Compare commits

..
Author SHA1 Message Date
shamoon 2d9ddf3dd0 Continue processing docs when wone WF run fails 2026-11-11 21:54:15 -08:00
shamoon d97c212a0f More enabled tests and tighten the view one 2026-11-11 21:50:20 -08:00
shamoon a2d31f9b59 Only enabled! 2026-11-11 21:42:02 -08:00
shamoon 62648fffe8 Fix migration 2026-11-06 14:15:05 -08:00
shamoon b0c8ce25fd docs 2026-11-06 14:14:29 -08:00
shamoon 3c4f5d8959 Add to doc details 2026-11-06 14:14:29 -08:00
shamoon 85c52cc27f tests 2026-11-06 14:14:29 -08:00
shamoon 9b3d484e24 Bulk run workflow + dialog 2026-11-06 14:14:28 -08:00
shamoon 3a5ba814b6 frotnend use the manual filter 2026-11-06 14:14:28 -08:00
shamoon 26552f54b4 Ah, need a backend filter for wf type 2026-11-06 14:14:28 -08:00
shamoon de9d7e54bd WorkflowTriggerType.Manual 2026-11-06 14:14:27 -08:00
shamoon a5c460a567 Add some backend tests for the manual wf 2026-11-06 14:14:27 -08:00
shamoon cfca79e7fc api bulk edit run_workflow 2026-11-06 14:14:27 -08:00
shamoon 9d59f3aaa2 run_workflows task= with filter_documents option 2026-11-06 14:14:26 -08:00
shamoon 332c308f84 Add skip_matching to run_workflows 2026-11-06 14:14:26 -08:00
shamoon a0b8da7687 Add manual trigger 2026-11-06 14:14:23 -08:00
83 changed files with 1427 additions and 2476 deletions

No files matched your search

@@ -1,23 +1,5 @@
title: "[Feature Request] "
body:
- type: markdown
attributes:
value: |
#### Use of AI tools
If this request was generated, in whole or in part, by an AI tool or agent you **must** say so using the field below. Failing to clearly disclose this is a violation of our [Code of Conduct](https://github.com/paperless-ngx/paperless-ngx/blob/main/CODE_OF_CONDUCT.md).
The project prefers thoughtful, well-described ideas to AI-generated content. Please describe the problem, your use case, the desired behavior, and who would benefit.
- type: dropdown
id: ai-usage
attributes:
label: AI usage
description: Was any part of this request written by an AI tool or agent?
options:
- No AI tools were used
- Yes, AI tools helped write this request
validations:
required: true
- type: textarea
id: description
attributes:
-16
View File
@@ -1,21 +1,5 @@
title: "[Support] "
body:
- type: markdown
attributes:
value: |
#### Use of AI tools
If this post was generated, in whole or in part, by an AI tool or agent you **must** say so using the field below. Failing to clearly disclose this is a violation of our [Code of Conduct](https://github.com/paperless-ngx/paperless-ngx/blob/main/CODE_OF_CONDUCT.md).
- type: dropdown
id: ai-usage
attributes:
label: AI usage
description: Was any part of this post written by an AI tool or agent?
options:
- No AI tools were used
- Yes, AI tools helped write this post
validations:
required: true
- type: textarea
id: description
attributes:
+4 -4
View File
@@ -38,7 +38,7 @@ repos:
- json
# See https://github.com/prettier/prettier/issues/15742 for the fork reason
- repo: https://github.com/rbubley/mirrors-prettier
rev: 'v3.9.10'
rev: 'v3.9.9'
hooks:
- id: prettier
types_or:
@@ -46,16 +46,16 @@ repos:
- ts
- markdown
additional_dependencies:
- prettier@3.9.10
- prettier@3.9.9
- 'prettier-plugin-organize-imports@4.3.0'
# Python hooks
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.17.2
rev: v0.16.9
hooks:
- id: ruff-check
- id: ruff-format
- repo: https://github.com/tox-dev/pyproject-fmt
rev: "v2.30.3"
rev: "v2.29.4"
hooks:
- id: pyproject-fmt
additional_dependencies: [tomli]
-44
View File
@@ -1,44 +0,0 @@
# Guidance for AI agents
Read [CONTRIBUTING.md](CONTRIBUTING.md) before working on this repository, especially [Use of AI Tools](CONTRIBUTING.md#use-of-ai-tools). These contribution rules apply to all agent-assisted work.
## Contribution boundaries
- Do not open or submit pull requests, including "on behalf of" a human. The project does not accept PRs from AI agents. Prepare local changes for a human to review, verify, and submit personally.
- The project does not accept PRs that are entirely or mostly AI-derived. Human submission and disclosure do not override this rule.
- AI-generated content must be clearly disclosed as described in the contributing guidelines. When handing work back to a human, identify what was generated or changed with AI assistance so they can disclose it accurately.
- Trust matters. Encourage first-time contributors to discuss their proposed change before preparing a PR, especially for large changes or changes involving AI-generated code. New features and enhancements should almost always target an existing feature request with community interest and discussion.
- The project prefers thoughtful, well-described ideas to AI-generated pull requests. Help contributors describe their problem, use case, desired behavior, and who would benefit. An implementation is not required for a valuable contribution, and a flawed implementation can obscure a useful idea before it receives proper discussion.
- AI tools have lowered the effort needed to produce a PR, but reviewing and maintaining it still requires substantial work. Encourage discussion of the idea before investing in implementation and review.
- Issue reports must describe observed behavior, reproduction steps, and relevant logs. Do not add speculative root causes, code analysis, or proposed fixes to agent-generated reports.
## Security reports
- AI-assisted security reports must meet the legitimacy and verification requirements in [SECURITY.md](SECURITY.md#ai-assisted-submissions). A human must verify a real Paperless-ngx code path, concrete reproduction steps, affected versions, and realistic security impact before submission.
- Account for existing permission checks, configuration requirements, and trust boundaries. Speculation, scanner output, or concerns about third-party software or deployment choices without a demonstrated Paperless-ngx vulnerability do not meet this threshold and may be closed without extended discussion.
- Disclose AI assistance and use the private vulnerability reporting channel specified in [SECURITY.md](SECURITY.md#reporting-a-vulnerability).
## Working on changes
- Read the relevant code, tests, and documentation before editing. Paperless-ngx is complex: trace interactions between affected components rather than assuming a change is isolated.
- Keep changes focused on the requested work and follow existing patterns. Avoid unrelated refactoring, formatting, dependency updates, or generated-file changes.
- Consider effects on permissions, document processing, background tasks, storage, and the API when those areas are involved.
- Add or update meaningful tests for behavior changes and regressions. Do not weaken tests or checks to make a change pass.
- Update user or developer documentation when behavior or setup changes.
- Keep credentials and private documents out of commits, fixtures, logs, and examples.
## Repository and validation
- `src/`: Django back end.
- `src-ui/`: Angular front end, managed with `pnpm`.
- `docs/`: Documentation.
- `docker/` and `scripts/`: Deployment and development tooling.
- Base functional changes on `dev`; documentation-only changes may target `main`.
Follow [the development documentation](docs/development.md) for environment setup and testing. Common checks after setup are:
- Back end: run `uv run pytest` from `src/` (use a test path to focus on the affected area).
- Front end: run `pnpm test` and `pnpm lint` from `src-ui/`.
- Formatting and other pre-commit checks: run `uv run prek run --files <changed-files>` from the repository root.
When handing work back, summarize the changes, the checks actually run and their results, and any remaining uncertainty. Clearly state when a check could not be run; do not claim verification that did not happen. The human contributor must understand and independently verify the contribution before submitting it.
+3 -16
View File
@@ -4,18 +4,6 @@ If you feel like contributing to the project, please do! Bug fixes and improveme
⚠️ Please note: Pull requests that implement a new feature or enhancement _should almost always target an existing feature request_ with evidence of community interest and discussion. This is in order to balance the work of implementing and maintaining new features / enhancements. Pull requests that are opened without meeting this requirement may not be merged.
## Trust and collaboration
Paperless-ngx places a high value on trust between contributors and maintainers. If you have not contributed before, consider starting with a discussion before preparing a pull request, especially for a large contribution or one involving AI-generated code.
Paperless-ngx is complex, and understanding how its components interact takes time. A change that appears correct in isolation can have consequences elsewhere in the application. Contributors should understand the code they submit, explain how it fits into the project, and work with maintainers through review. Starting with smaller, focused contributions can help build that understanding and trust.
We prefer thoughtful, well-described ideas to AI-generated pull requests. You do not need to supply an implementation for your contribution to be valuable. Explain the problem you have encountered, your use case, the behavior you would like to see, and who would benefit. Concrete examples, alternatives you have considered, and relevant constraints help the community discuss and refine the idea.
AI tools have lowered the effort needed to produce a pull request, but evaluating one still requires substantial maintainer time. Maintainers still need to establish whether the implementation is correct, fits the project, and can be maintained over time.
A flawed implementation can also obscure a good idea: discussion becomes focused on problems with the generated code before the community has considered whether the underlying proposal is useful. Starting with the idea gives it a fair discussion on its own merits and lets contributors and maintainers agree on an approach before investing in implementation and review.
If you want to implement something big:
- As above, please start with a discussion! Maybe something similar is already in development and we can make it happen together.
@@ -73,10 +61,9 @@ This project does not specifically prohibit the use of AI tools or agents _durin
1. Any content — code, pull request descriptions, issue reports or comments — that was generated in whole or in part by an AI tool or agent **must be clearly disclosed as such**. Failing to clearly disclose this is considered a violation of our [Code of Conduct](CODE_OF_CONDUCT.md) and may result in your contributions being closed and further action being taken.
2. AI-generated code must not violate copyright protections.
3. We will not accept PRs opened or submitted by AI agents, including agents acting "on behalf of" a human. A human contributor must personally review, verify, and submit the contribution and take responsibility for it throughout review.
4. We will not accept PRs that are entirely or mostly AI-derived.
5. Issue reports must describe _observed behavior only_: what happened, how to reproduce it and the relevant logs. Reports generated by an AI agent should not include code analysis, speculation about the root cause or suggest fixes.
6. Contributors are responsible for everything they submit, regardless of how it was produced. Please do not open issues or PRs you have not verified yourself.
3. We will not accept PRs that are entirely or mostly AI-derived.
4. Issue reports must describe _observed behavior only_: what happened, how to reproduce it and the relevant logs. Reports generated by an AI agent should not include code analysis, speculation about the root cause or suggest fixes.
5. Contributors are responsible for everything they submit, regardless of how it was produced. Please do not open issues or PRs you have not verified yourself.
# Translating Paperless-ngx
-3
View File
@@ -56,9 +56,6 @@ We may close reports that are:
The following are not generally considered vulnerabilities unless accompanied by a concrete, reproducible impact in Paperless-ngx:
- password length inferred from masked mail-account passwords by users who already have permission to view the account, without disclosure of the password or token itself
- disclosure limited to document IDs, existence, page counts, checksums, or modification/deletion timestamps, without access to document contents or an unintended privileged action
- authenticated access to non-secret application settings, such as service endpoint URLs, model names, or configured environment variable names, without disclosure of credentials or secret values
- large uploads or resource usage that do not bypass documented limits or privileges
- IDOR / access control claims regarding the ability to attach an un-viewable object to a document. This is expected behavior.
- claims based solely on the presence of a library, framework feature or code pattern without a working exploit
-2
View File
@@ -565,8 +565,6 @@ Specify `rebuild` to build the index from scratch from all documents in the data
this the first time you enable the feature, or after changing the embedding backend or
model.
When using the `huggingface` embedding backend, a model that is already downloaded is loaded from the local cache without contacting the Hugging Face Hub. Only `rebuild` checks the Hub for a newer revision of the model, so a model update is picked up together with re-embedding every document. If the model is not cached yet, it is downloaded on first use.
Specify `update` to incrementally index new and changed documents. This is what the
scheduled task runs.
+11 -22
View File
@@ -501,20 +501,20 @@ See the [Django project documentation on the settings](https://docs.djangoprojec
#### [`PAPERLESS_ALLOWED_HOSTS=<comma-separated-list>`](#PAPERLESS_ALLOWED_HOSTS) {#PAPERLESS_ALLOWED_HOSTS}
: If you expose Paperless to the internet, set `PAPERLESS_URL` to its public URL
or list its public hostname in `PAPERLESS_ALLOWED_HOSTS` to help protect against
HTTP Host header attacks.
: If you're planning on putting Paperless on the open internet, then
you really should set this value to the domain name you're using.
Failing to do so leaves you open to HTTP host header attacks.
You can read more about this in [the Django project's documentation](https://docs.djangoproject.com/en/4.1/topics/security/#host-header-validation)
Separate multiple hostnames with commas, as in `example.com,www.example.com`.
Just remember that this is a comma-separated list, so
"example.com" is fine, as is "example.com,www.example.com", but
NOT " example.com" or "example.com,"
- With neither setting configured, Paperless accepts any host.
- With only `PAPERLESS_URL` set, Paperless accepts the hostname from that URL
and `localhost` (used by the Docker health check).
- `PAPERLESS_ALLOWED_HOSTS` adds hosts to `localhost` and the hostname from
`PAPERLESS_URL`, if set. A blank list adds no hosts.
- With `PAPERLESS_ALLOWED_HOSTS=*`, Paperless accepts any host, even when
`PAPERLESS_URL` is set.
Can also be set using PAPERLESS_URL (see above).
"localhost" is always allowed for docker healthcheck
Defaults to "\*", which is all hosts.
#### [`PAPERLESS_CORS_ALLOWED_HOSTS=<comma-separated-list>`](#PAPERLESS_CORS_ALLOWED_HOSTS) {#PAPERLESS_CORS_ALLOWED_HOSTS}
@@ -873,14 +873,6 @@ If both the [PAPERLESS_ACCOUNT_DEFAULT_GROUPS](#PAPERLESS_ACCOUNT_DEFAULT_GROUPS
Defaults to 1209600 (2 weeks)
#### [`PAPERLESS_SESSION_CLEANUP_TASK_CRON=<cron expression>`](#PAPERLESS_SESSION_CLEANUP_TASK_CRON) {#PAPERLESS_SESSION_CLEANUP_TASK_CRON}
: Configures the schedule to remove expired sessions from the session store.
Django does not prune these automatically, so without this they would
accumulate indefinitely.
Defaults to `0 3 * * *`, once per day at 03:00.
#### [`PAPERLESS_TOKEN_THROTTLE_RATE=<rate>`](#PAPERLESS_TOKEN_THROTTLE_RATE) {#PAPERLESS_TOKEN_THROTTLE_RATE}
: Rate limit for the API token authentication endpoint (`/api/token/`), used to mitigate brute-force login attempts.
@@ -1157,9 +1149,6 @@ they use underscores instead of dashes.
be changed by trusted users. This applies to the `AppConfig` permission as well,
which allows setting these arguments from the UI.
Arguments paperless manages itself (`input_file_or_options`, `output_file`
and `sidecar`) are rejected by the API and ignored if set here.
Specify arguments as a JSON dictionary. Keep note of lower case
booleans and double quoted parameter names and strings. Examples:
+30 -5
View File
@@ -493,7 +493,7 @@ fields and permissions, which will be merged.
#### Types {#workflow-trigger-types}
Currently, there are four events that correspond to workflow trigger 'types':
Currently, there are five events that correspond to workflow trigger 'types':
1. **Consumption Started**: _before_ a document is consumed, so events can include filters by source (mail, consumption
folder or API), file path, file name, mail rule
@@ -505,8 +505,10 @@ Currently, there are four events that correspond to workflow trigger 'types':
4. **Scheduled**: a scheduled trigger that can be used to run workflows at a specific time. The date used can be either the document
added, created, updated date or you can specify a (date) custom field. You can also specify a day offset from the date (positive
offsets will trigger after the date, negative offsets will trigger before).
5. **Manual**: a trigger that only runs when you explicitly choose to run the workflow against one or more existing documents. See
[running a workflow manually](#running-a-workflow-manually) below.
The following flow diagram illustrates the four document trigger types:
The following flow diagram illustrates the document trigger types:
```mermaid
flowchart TD
@@ -526,6 +528,10 @@ flowchart TD
matching
trigger(s)"}
manual{"Matching
'Manual'
trigger(s)"}
A[New Document] --> consumption
consumption --> |Yes| C[Workflow Actions Run]
consumption --> |No| D
@@ -543,6 +549,10 @@ flowchart TD
scheduled --> |Yes| N[Workflow Actions Run]
scheduled --> |No| O[Document Saved]
N --> O
P[User Runs Workflow] --> manual
manual --> |Yes| Q[Workflow Actions Run]
manual --> |No| R[Document Saved]
Q --> R
```
#### Filters {#workflow-trigger-filters}
@@ -554,9 +564,9 @@ Workflows allow you to filter by:
- File path, including wildcards. Note that enabling `PAPERLESS_CONSUMER_RECURSIVE` would allow, for
example, automatically assigning documents to different owners based on the upload directory.
- Mail rule. Choosing this option will force 'mail fetch' to be the workflow source.
- Content matching (`Added`, `Updated` and `Scheduled` triggers only). Filter document content using the matching settings.
- Content matching (`Added`, `Updated`, `Scheduled` and `Manual` triggers only). Filter document content using the matching settings.
There are also 'advanced' filters available for `Added`, `Updated` and `Scheduled` triggers:
There are also 'advanced' filters available for `Added`, `Updated`, `Scheduled` and `Manual` triggers:
- Any Tags: Filter for documents with any of the specified tags.
- All Tags: Filter for documents with all of the specified tags.
@@ -569,6 +579,21 @@ There are also 'advanced' filters available for `Added`, `Updated` and `Schedule
- Not Storage paths: Filter documents without any of these storage paths.
- Custom field query: Filter documents with a custom field query (the same as used for the document list filters).
#### Running a workflow manually {#running-a-workflow-manually}
A workflow with a **Manual** trigger does not run automatically. Instead, it can be run on demand against existing
documents from the web UI:
- From the document list, select one or more documents and choose **Run workflow** from the "Actions" menu.
- From an individual document, choose **Run workflow** from the "Actions" menu.
You will be asked which workflow to run. By default the workflow's trigger filters still apply, so documents that do
not match are skipped. You can instead choose to **run on all selected documents**, which ignores the trigger filters
and runs the workflow's actions on every selected document.
Manual runs happen in the background. As with scheduled workflows, the document list is not refreshed automatically when
they finish, so you may need to reload the list to see the results.
### Workflow Actions
#### Types {#workflow-action-types}
@@ -649,7 +674,7 @@ new version of the same document, this time with normal OCR/text extraction appl
a password-protected file added with this trigger will briefly exist as an un-OCR'd version before
the properly processed version is created.
**Document Added**, **Document Updated**, **Scheduled**: these triggers run against a document that
**Document Added**, **Document Updated**, **Scheduled**, **Manual**: these triggers run against a document that
already exists, so password removal happens immediately: the decrypted content is queued for
consumption as a new version right away. Note that if the document's initial consumption also
happened while it was still encrypted, that original version will likewise be missing OCR text.
+12 -7
View File
@@ -86,9 +86,9 @@ mariadb = [
"mysqlclient>=2.2.7,<2.4",
]
postgres = [
"psycopg[c,pool]==3.3.8",
"psycopg[c,pool]==3.3.4",
# Direct dependency for proper resolution of the pre-built wheels
"psycopg-c==3.3.8",
"psycopg-c==3.3.4",
"psycopg-pool==3.3.3",
]
webserver = [
@@ -158,10 +158,10 @@ package = false
[tool.uv.sources]
# Markers are chosen to select these almost exclusively when building the Docker image
psycopg-c = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.14'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.14'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.12'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl", marker = "sys_platform == 'linux' and platform_machine == 'x86_64' and python_version == '3.14'" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl", marker = "sys_platform == 'linux' and platform_machine == 'aarch64' and python_version == '3.14'" },
]
torch = [
{ index = "pytorch-cpu" },
@@ -212,7 +212,12 @@ extend-select = [
"UP", # https://docs.astral.sh/ruff/rules/#pyupgrade-up
"W", # https://docs.astral.sh/ruff/rules/#pycodestyle-e-w
]
ignore = [ "DJ001", "ICN001", "PLC0415", "RUF012", "SIM105" ]
ignore = [
"DJ001",
"PLC0415",
"RUF012",
"SIM105",
]
# Migrations
per-file-ignores."*/migrations/*.py" = [
"E501",
+136 -178
View File
File diff suppressed because it is too large. Load diff
@@ -7,15 +7,10 @@ import {
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormsModule } from '@angular/forms'
import { By } from '@angular/platform-browser'
import {
ActivatedRoute,
convertToParamMap,
ParamMap,
Router,
} from '@angular/router'
import { Router } from '@angular/router'
import { RouterTestingModule } from '@angular/router/testing'
import { NgbModal, NgbModalRef, NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { BehaviorSubject, of, throwError } from 'rxjs'
import { of, throwError } from 'rxjs'
import { routes } from 'src/app/app-routing.module'
import {
PaperlessTask,
@@ -164,7 +159,6 @@ const sectionCountResponse = {
}
describe('TasksComponent', () => {
let queryParams: BehaviorSubject<ParamMap>
let component: TasksComponent
let fixture: ComponentFixture<TasksComponent>
let tasksService: TasksService
@@ -175,7 +169,6 @@ describe('TasksComponent', () => {
let toastService: ToastService
beforeEach(async () => {
queryParams = new BehaviorSubject(convertToParamMap({}))
TestBed.configureTestingModule({
imports: [
NgbModule,
@@ -188,17 +181,6 @@ describe('TasksComponent', () => {
ConfirmDialogComponent,
],
providers: [
{
provide: ActivatedRoute,
useValue: {
queryParamMap: queryParams.asObservable(),
snapshot: {
get queryParamMap() {
return queryParams.value
},
},
},
},
{
provide: PermissionsService,
useValue: {
@@ -237,11 +219,7 @@ describe('TasksComponent', () => {
})
// answers task list requests with the fixture tasks matching their status filter
function flushTaskLists(
pageSize: number,
count?: number,
filteredTasks: PaperlessTask[] = tasks
) {
function flushTaskLists(pageSize: number, count?: number) {
httpTestingController
.match(
(req) =>
@@ -250,9 +228,7 @@ describe('TasksComponent', () => {
)
.forEach((req) => {
const statuses = req.request.params.getAll('status')
const results = filteredTasks.filter((task) =>
statuses.includes(task.status)
)
const results = tasks.filter((task) => statuses.includes(task.status))
req.flush({ count: count ?? results.length, results })
})
}
@@ -262,71 +238,6 @@ describe('TasksComponent', () => {
flushTaskLists(25, count)
}
it('consumes a task link on load into the normal resettable Task ID filter', () => {
jest
.spyOn(tasksService, 'list')
.mockReturnValue(of({ count: 0, results: [] }))
jest
.spyOn(tasksService, 'statusCounts')
.mockReturnValue(of(sectionCountResponse))
jest.spyOn(router, 'navigate').mockResolvedValue(true)
queryParams.next(convertToParamMap({ task_id: 'linked-task' }))
fixture.destroy()
fixture = TestBed.createComponent(TasksComponent)
component = fixture.componentInstance
fixture.detectChanges()
expect(router.navigate).toHaveBeenCalledTimes(1)
expect(router.navigate).toHaveBeenCalledWith(
[],
expect.objectContaining({
queryParams: { task_id: null },
replaceUrl: true,
})
)
expect(component.filterTargetName).toBe('Task ID')
expect(component.filterText).toBe('linked-task')
expect(component.isFiltered).toBe(true)
component.setSection(TaskSection.InProgress)
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
task_id: 'linked-task',
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
})
component.resetFilters()
expect(component.filterText).toBe('')
expect(component.isFiltered).toBe(false)
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
acknowledged: false,
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
})
expect(router.navigate).toHaveBeenCalledTimes(1)
})
it('supports manually filtering by Task ID and clearing the text filter', () => {
jest
.spyOn(tasksService, 'list')
.mockReturnValue(of({ count: 0, results: [] }))
jest
.spyOn(tasksService, 'statusCounts')
.mockReturnValue(of(sectionCountResponse))
component.setSection(TaskSection.InProgress)
component.setFilterTarget(TaskFilterTargetID.TaskId)
component.filterText = tasks[0].task_id
jest.advanceTimersByTime(150)
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
task_id: tasks[0].task_id,
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
})
component.pagedTasks.set(tasks)
expect(component.tasksForSection(TaskSection.NeedsAttention)).toEqual([
tasks[0],
])
component.resetFilter()
expect(tasksService.list).toHaveBeenLastCalledWith(1, 25, {
acknowledged: false,
status: [PaperlessTaskStatus.Pending, PaperlessTaskStatus.Started],
})
})
it('should display task sections with counts', () => {
expect(component.selectedSection).toBe(TaskSection.All)
expect(component.selectedTaskType).toBeNull()
@@ -860,7 +771,6 @@ describe('TasksComponent', () => {
input.nativeElement.value = '191092'
input.nativeElement.dispatchEvent(new Event('input'))
jest.advanceTimersByTime(150)
flushTaskLists(10, undefined, [tasks[1]])
fixture.detectChanges()
expect(component.filterText).toEqual('191092')
expect(component.tasksForSection(TaskSection.NeedsAttention)).toHaveLength(
@@ -872,7 +782,6 @@ describe('TasksComponent', () => {
component.setSection(TaskSection.InProgress)
component.filterText = 'system'
jest.advanceTimersByTime(150)
flushTaskLists(25, undefined, [tasks[6]])
expect(component.tasksForSection(TaskSection.InProgress)).toHaveLength(1)
expect(component.tasksForSection(TaskSection.InProgress)[0].task_type).toBe(
@@ -906,7 +815,6 @@ describe('TasksComponent', () => {
input.nativeElement.value = 'duplicate'
input.nativeElement.dispatchEvent(new Event('input'))
jest.advanceTimersByTime(150)
flushTaskLists(25, undefined, [tasks[0], tasks[1]])
fixture.detectChanges()
expect(component.filterText).toEqual('duplicate')
expect(component.tasksForSection(TaskSection.NeedsAttention)).toHaveLength(
@@ -914,53 +822,6 @@ describe('TasksComponent', () => {
)
})
it('should show server-matched names when the displayed source is translated', () => {
component.setSection(TaskSection.InProgress)
flushTaskLists(25)
component.filterText = 'email'
jest.advanceTimersByTime(150)
const request = httpTestingController.expectOne(
(req) =>
req.url === `${environment.apiBaseUrl}tasks/` &&
req.params.get('name') === 'email' &&
req.params.get('page_size') === '25'
)
const translatedTask = {
...tasks[5],
trigger_source_display: 'E-Mail',
}
request.flush({ count: 1, results: [translatedTask] })
fixture.detectChanges()
expect(component.totalTasks()).toBe(1)
expect(component.visibleTasks).toEqual([translatedTask])
})
it('should show server-matched duplicates when the displayed result is translated', () => {
component.setSection(TaskSection.NeedsAttention)
flushTaskLists(25)
jest
.spyOn(component, 'taskResultMessage')
.mockReturnValue('Duplikat von Document #311')
component.setFilterTarget(TaskFilterTargetID.Result)
component.filterText = 'duplicate'
jest.advanceTimersByTime(150)
const request = httpTestingController.expectOne(
(req) =>
req.url === `${environment.apiBaseUrl}tasks/` &&
req.params.get('result') === 'duplicate' &&
req.params.get('page_size') === '25'
)
request.flush({ count: 1, results: [tasks[1]] })
fixture.detectChanges()
expect(component.totalTasks()).toBe(1)
expect(component.visibleTasks).toEqual([tasks[1]])
expect(fixture.nativeElement.textContent).toContain('191092.pdf')
})
it('should prefer explicit reason in the result message', () => {
expect(
component.taskResultMessage({
@@ -1,7 +1,7 @@
import { JsonPipe, NgTemplateOutlet } from '@angular/common'
import { Component, inject, OnDestroy, OnInit, signal } from '@angular/core'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { ActivatedRoute, Router, RouterLink } from '@angular/router'
import { Router, RouterLink } from '@angular/router'
import {
NgbCollapseModule,
NgbDropdownModule,
@@ -47,13 +47,11 @@ export enum TaskSection {
export enum TaskFilterTargetID {
Name,
Result,
TaskId,
}
const FILTER_TARGETS = [
{ id: TaskFilterTargetID.Name, name: $localize`Name` },
{ id: TaskFilterTargetID.Result, name: $localize`Result` },
{ id: TaskFilterTargetID.TaskId, name: $localize`Task ID` },
]
const SECTION_LABELS = {
@@ -163,7 +161,6 @@ export class TasksComponent
tasksService = inject(TasksService)
private modalService = inject(NgbModal)
private readonly router = inject(Router)
private readonly route = inject(ActivatedRoute)
private readonly toastService = inject(ToastService)
readonly TaskSection = TaskSection
@@ -278,17 +275,6 @@ export class TasksComponent
}
ngOnInit() {
const taskId = this.route.snapshot.queryParamMap.get('task_id')
if (taskId) {
this.filterTargetID = TaskFilterTargetID.TaskId
this._filterText = taskId
this.router.navigate([], {
relativeTo: this.route,
queryParams: { task_id: null },
queryParamsHandling: 'merge',
replaceUrl: true,
})
}
this.tasksService.reload()
this.reloadPage()
timer(5000, 5000)
@@ -650,12 +636,17 @@ export class TasksComponent
const query = this._filterText.toLowerCase()
if (this.filterTargetID === TaskFilterTargetID.TaskId) {
return task.task_id.toLowerCase() === query
if (this.filterTargetID == TaskFilterTargetID.Name) {
return [
this.taskDisplayName(task),
task.task_type_display,
task.trigger_source_display,
]
.filter(Boolean)
.some((value) => value.toLowerCase().includes(query))
}
// Text filtering now handled API-side
return true
return this.taskResultMessage(task)?.toLowerCase().includes(query) ?? false
}
private tasksForOptionCounts({
@@ -715,15 +706,9 @@ export class TasksComponent
}
if (this._filterText.length) {
const parameter = {
[TaskFilterTargetID.Name]: 'name',
[TaskFilterTargetID.Result]: 'result',
[TaskFilterTargetID.TaskId]: 'task_id',
}[this.filterTargetID]
params[parameter] = this._filterText
if (this.filterTargetID === TaskFilterTargetID.TaskId) {
delete params.acknowledged
}
params[
this.filterTargetID === TaskFilterTargetID.Name ? 'name' : 'result'
] = this._filterText
}
return params
@@ -11,10 +11,8 @@ import { ActivatedRoute, Router } from '@angular/router'
import { RouterTestingModule } from '@angular/router/testing'
import { NgbModal, NgbModalModule, NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { provideUiTour } from 'ngx-ui-tour-ng-bootstrap'
import { of, Subject, throwError } from 'rxjs'
import { of, throwError } from 'rxjs'
import { routes } from 'src/app/app-routing.module'
import { BulkOperationStatus } from 'src/app/data/bulk-operation'
import { PaperlessTaskStatus } from 'src/app/data/paperless-task'
import { SavedView } from 'src/app/data/saved-view'
import { HideableSidebarItemID, SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
@@ -23,11 +21,10 @@ import {
DjangoMessageLevel,
DjangoMessagesService,
} from 'src/app/services/django-messages.service'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
import {
PermissionsService,
PermissionType,
PermissionsService,
} from 'src/app/services/permissions.service'
import { RemoteVersionService } from 'src/app/services/rest/remote-version.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
@@ -98,7 +95,6 @@ describe('AppFrameComponent', () => {
let savedViewSpy
let modalService: NgbModal
let maybeRefreshSpy
let bulkEvents: Subject<BulkOperationStatus>
let tasksService: TasksService
beforeEach(async () => {
@@ -186,10 +182,6 @@ describe('AppFrameComponent', () => {
savedViewSpy = jest.spyOn(savedViewService, 'reload')
maybeRefreshSpy = jest.spyOn(savedViewService, 'maybeRefreshDocumentCounts')
bulkEvents = new Subject<BulkOperationStatus>()
jest
.spyOn(TestBed.inject(WebsocketStatusService), 'onBulkOperationStatus')
.mockReturnValue(bulkEvents.asObservable())
fixture = TestBed.createComponent(AppFrameComponent)
component = fixture.componentInstance
@@ -198,78 +190,6 @@ describe('AppFrameComponent', () => {
fixture.detectChanges()
})
it('refreshes and notifies on completion without client-side task tracking', () => {
const list = TestBed.inject(DocumentListViewService)
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
const tasksReload = jest
.spyOn(tasksService, 'reload')
.mockImplementation(() => {})
const success = jest.spyOn(toastService, 'showSuccess')
const refresh = jest
.spyOn(openDocumentsService, 'refreshDocument')
.mockImplementation(() => {})
jest
.spyOn(openDocumentsService, 'getOpenDocuments')
.mockReturnValue([document])
tasksReload.mockClear()
maybeRefreshSpy.mockClear()
bulkEvents.next({
task_id: 'bulk-task',
operation: 'set_permissions',
status: PaperlessTaskStatus.Started,
})
expect(reload).not.toHaveBeenCalled()
bulkEvents.next({
task_id: 'bulk-task',
operation: 'set_permissions',
status: PaperlessTaskStatus.Success,
})
expect(reload).toHaveBeenCalledTimes(1)
expect(reload).toHaveBeenCalledWith(null, false)
expect(refresh).toHaveBeenCalledWith(document.id)
expect(maybeRefreshSpy).toHaveBeenCalledTimes(1)
expect(tasksReload).toHaveBeenCalledTimes(1)
expect(success).toHaveBeenCalledWith('Bulk operation completed.')
fixture.destroy()
bulkEvents.next({
task_id: 'bulk-task',
operation: 'set_permissions',
status: PaperlessTaskStatus.Success,
})
expect(reload).toHaveBeenCalledTimes(1)
})
it('refreshes and reports failed or revoked bulk operations', () => {
const list = TestBed.inject(DocumentListViewService)
const reload = jest.spyOn(list, 'reload').mockImplementation(() => {})
jest.spyOn(list, 'reduceSelectionToFilter').mockImplementation(() => {})
jest.spyOn(tasksService, 'reload').mockImplementation(() => {})
const error = jest.spyOn(toastService, 'showError')
const success = jest.spyOn(toastService, 'showSuccess')
for (const status of [
PaperlessTaskStatus.Failure,
PaperlessTaskStatus.Revoked,
]) {
bulkEvents.next({
task_id: 'bulk-task',
operation: 'set_permissions',
status,
error:
status === PaperlessTaskStatus.Revoked
? 'Task expired before execution.'
: 'Could not update',
})
}
expect(reload).toHaveBeenCalledTimes(2)
expect(error).toHaveBeenCalledWith(expect.any(String), 'Could not update')
expect(error).toHaveBeenCalledWith(
'Bulk operation revoked.',
'Task expired before execution.'
)
expect(success).not.toHaveBeenCalled()
})
it('should initialize the saved view service', () => {
expect(savedViewSpy).toHaveBeenCalled()
})
@@ -8,14 +8,12 @@ import {
import { NgClass } from '@angular/common'
import {
Component,
DestroyRef,
HostListener,
inject,
OnInit,
signal,
viewChild,
} from '@angular/core'
import { takeUntilDestroyed } from '@angular/core/rxjs-interop'
import { FormsModule } from '@angular/forms'
import { ActivatedRoute, Router, RouterModule } from '@angular/router'
import {
@@ -26,11 +24,10 @@ import {
NgbPopoverModule,
} from '@ng-bootstrap/ng-bootstrap'
import { TourNgBootstrap } from 'ngx-ui-tour-ng-bootstrap'
import { filter, Observable } from 'rxjs'
import { Observable } from 'rxjs'
import { first } from 'rxjs/operators'
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
import { Document } from 'src/app/data/document'
import { PaperlessTaskStatus } from 'src/app/data/paperless-task'
import { SavedView } from 'src/app/data/saved-view'
import {
CollapsibleSection,
@@ -44,7 +41,6 @@ import {
DjangoMessageLevel,
DjangoMessagesService,
} from 'src/app/services/django-messages.service'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
import {
PermissionAction,
@@ -105,8 +101,6 @@ export class AppFrameComponent
extends ComponentWithPermissions
implements OnInit, ComponentCanDeactivate
{
private destroyRef = inject(DestroyRef)
private documentList = inject(DocumentListViewService)
router = inject(Router)
private activatedRoute = inject(ActivatedRoute)
private openDocumentsService = inject(OpenDocumentsService)
@@ -170,46 +164,6 @@ export class AppFrameComponent
}
ngOnInit(): void {
this.websocketStatusService
.onBulkOperationStatus()
.pipe(
filter(({ status }) =>
[
PaperlessTaskStatus.Success,
PaperlessTaskStatus.Failure,
PaperlessTaskStatus.Revoked,
].includes(status)
),
takeUntilDestroyed(this.destroyRef)
)
.subscribe(({ status, error }) => {
this.documentList.reload(null, false)
this.documentList.reduceSelectionToFilter()
// maybe someday we can optimize to only refresh affected documents
this.openDocumentsService.getOpenDocuments().forEach((doc) => {
this.openDocumentsService.refreshDocument(doc.id)
})
this.savedViewService.maybeRefreshDocumentCounts()
if (
this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.PaperlessTask
)
) {
this.tasksService.reload()
}
if (status === PaperlessTaskStatus.Success) {
this.toastService.showSuccess($localize`Bulk operation completed.`)
} else if (status === PaperlessTaskStatus.Revoked) {
this.toastService.showError($localize`Bulk operation revoked.`, error)
} else {
this.toastService.showError(
$localize`Bulk operation failed. Some changes may already have been applied.`,
error
)
}
})
this.lastScrollY = window.scrollY
this.detectClassicScrollbars()
@@ -42,16 +42,11 @@
<span class="text-break">{{ message.content }}</span>
}
@if (message.role === 'assistant' && !message.isStreaming && message.references?.length) {
<div class="chat-references mt-2 pt-2 border-top">
<div class="d-flex align-items-center text-muted mb-1">
<span i18n>Sources</span>
@if (hiddenReferenceCount(message) > 0 && !expandedReferences().has(message)) {
<button type="button" class="badge rounded-pill bg-body-secondary text-body-secondary fw-normal border-0 ms-2" (click)="expandReferences(message)" i18n>+{{ hiddenReferenceCount(message) }} more</button>
}
</div>
@for (reference of visibleReferences(message); track reference.id) {
<a class="reference-link d-flex align-items-center lh-sm text-primary text-decoration-none py-1" [routerLink]="['/documents', reference.id]" [title]="reference.title">
<pngx-bs-icon width="0.8em" height="0.8em" name="file-text" class="me-2 flex-shrink-0"></pngx-bs-icon><span class="text-truncate">{{ reference.title }}</span>
<div class="chat-references list-group mt-3">
<div class="list-group-item text-muted small py-1" i18n>Sources</div>
@for (reference of message.references; track reference.id) {
<a class="list-group-item list-group-item-action d-flex align-items-center text-primary" [routerLink]="['/documents', reference.id]" [title]="reference.title">
<pngx-bs-icon width="0.9em" height="0.9em" name="file-text" class="me-2 flex-shrink-0"></pngx-bs-icon><span class="text-truncate">{{ reference.title }}</span>
</a>
}
</div>
@@ -12,10 +12,6 @@
display: none;
}
.message .flex-grow-1 {
min-width: 0;
}
.dropdown-item {
white-space: initial;
}
@@ -14,7 +14,6 @@ import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon
import { MarkdownPipe } from 'src/app/pipes/markdown.pipe'
import {
ChatMessage,
ChatReference,
ChatService,
parseChatResponse,
} from 'src/app/services/chat.service'
@@ -41,29 +40,8 @@ export class ChatComponent implements OnInit {
readonly input = signal('')
readonly documentId = signal<number>(undefined)
readonly slowResponse = signal(false)
readonly expandedReferences = signal<Set<ChatMessage>>(new Set())
public popperOptions = pngxPopperOptions
static readonly COLLAPSED_REFERENCE_COUNT = 2
public visibleReferences(message: ChatMessage): ChatReference[] {
if (this.expandedReferences().has(message)) {
return message.references
}
return message.references.slice(0, ChatComponent.COLLAPSED_REFERENCE_COUNT)
}
public hiddenReferenceCount(message: ChatMessage): number {
return Math.max(
0,
message.references.length - ChatComponent.COLLAPSED_REFERENCE_COUNT
)
}
public expandReferences(message: ChatMessage): void {
this.expandedReferences.update((expanded) => new Set(expanded).add(message))
}
private chatService: ChatService = inject(ChatService)
private router: Router = inject(Router)
@@ -118,6 +118,7 @@ import personFill from 'bootstrap-icons/icons/person-fill.svg'
import personLock from 'bootstrap-icons/icons/person-lock.svg'
import personSquare from 'bootstrap-icons/icons/person-square.svg'
import person from 'bootstrap-icons/icons/person.svg'
import playCircle from 'bootstrap-icons/icons/play-circle.svg'
import playFill from 'bootstrap-icons/icons/play-fill.svg'
import plusCircle from 'bootstrap-icons/icons/plus-circle.svg'
import plus from 'bootstrap-icons/icons/plus.svg'
@@ -279,6 +280,7 @@ export const icons: Record<string, string> = {
personFillLock,
personLock,
personSquare,
playCircle,
playFill,
plus,
plusCircle,
@@ -0,0 +1,32 @@
<div class="modal-header">
<h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
<button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
</button>
</div>
<div class="modal-body">
@if (message) {
<p class="text-break" [innerHTML]="message"></p>
}
<div class="mb-3">
<label class="form-label" for="runWorkflowSelect" i18n>Workflow</label>
<select class="form-select" id="runWorkflowSelect" [(ngModel)]="workflowId" (ngModelChange)="workflowSelected()">
<option [ngValue]="null" disabled i18n>Select a workflow</option>
@for (workflow of workflows; track workflow.id) {
<option [ngValue]="workflow.id">{{workflow.name}}</option>
}
</select>
</div>
<div class="form-check">
<input class="form-check-input" type="checkbox" id="runWorkflowForce" [(ngModel)]="force" />
<label class="form-check-label" for="runWorkflowForce" i18n>Run on all selected documents</label>
<div class="form-text" i18n>Ignore the workflow's trigger filters and run on every selected document.</div>
</div>
</div>
<div class="modal-footer">
<button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
<span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
</button>
<button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
{{btnCaption}}
</button>
</div>
@@ -0,0 +1,42 @@
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { RunWorkflowConfirmDialogComponent } from './run-workflow-confirm-dialog.component'
describe('RunWorkflowConfirmDialogComponent', () => {
let component: RunWorkflowConfirmDialogComponent
let fixture: ComponentFixture<RunWorkflowConfirmDialogComponent>
beforeEach(async () => {
TestBed.configureTestingModule({
providers: [NgbActiveModal],
imports: [RunWorkflowConfirmDialogComponent],
}).compileComponents()
fixture = TestBed.createComponent(RunWorkflowConfirmDialogComponent)
component = fixture.componentInstance
component.workflows = [{ id: 5, name: 'Manual workflow' } as any]
fixture.detectChanges()
})
it('should disable confirm until a workflow is selected and not force by default', () => {
expect(component.confirmButtonEnabled).toBeFalsy()
expect(component.force).toBeFalsy()
component.workflowId = 5
component.workflowSelected()
expect(component.confirmButtonEnabled).toBeTruthy()
component.workflowId = null
component.workflowSelected()
expect(component.confirmButtonEnabled).toBeFalsy()
})
it('should render an option per workflow', () => {
const options = fixture.nativeElement.querySelectorAll(
'#runWorkflowSelect option'
)
// placeholder option + one per workflow
expect(options).toHaveLength(2)
expect(options[1].textContent).toContain('Manual workflow')
})
})
@@ -0,0 +1,24 @@
import { Component } from '@angular/core'
import { FormsModule } from '@angular/forms'
import { Workflow } from 'src/app/data/workflow'
import { ConfirmDialogComponent } from '../confirm-dialog.component'
@Component({
selector: 'pngx-run-workflow-confirm-dialog',
templateUrl: './run-workflow-confirm-dialog.component.html',
imports: [FormsModule],
})
export class RunWorkflowConfirmDialogComponent extends ConfirmDialogComponent {
public workflows: Workflow[] = []
public workflowId: number = null
// i.e. ignore the workflow's trigger filters
public force: boolean = false
override confirmButtonEnabled = false
public workflowSelected(): void {
this.confirmButtonEnabled = !!this.workflowId
}
}
@@ -164,7 +164,7 @@
<pngx-input-text i18n-title title="Filter path" formControlName="filter_path" horizontal="true" i18n-hint hint="Apply to documents that match this path. Wildcards specified as * are allowed. Case-normalized." [error]="error?.filter_path"></pngx-input-text>
<pngx-input-select i18n-title title="Filter mail rule" [items]="mailRules()" horizontal="true" [allowNull]="true" formControlName="filter_mailrule" i18n-hint hint="Apply to documents consumed via this mail rule." [error]="error?.filter_mailrule"></pngx-input-select>
}
@if (formGroup.get('type').value === WorkflowTriggerType.DocumentAdded || formGroup.get('type').value === WorkflowTriggerType.DocumentUpdated || formGroup.get('type').value === WorkflowTriggerType.Scheduled) {
@if (formGroup.get('type').value === WorkflowTriggerType.DocumentAdded || formGroup.get('type').value === WorkflowTriggerType.DocumentUpdated || formGroup.get('type').value === WorkflowTriggerType.Scheduled || formGroup.get('type').value === WorkflowTriggerType.Manual) {
<pngx-input-select i18n-title title="Content matching algorithm" horizontal="true" [items]="getMatchingAlgorithms()" formControlName="matching_algorithm"></pngx-input-select>
@if (matchingPatternRequired(formGroup)) {
<pngx-input-text i18n-title title="Content matching pattern" horizontal="true" formControlName="match" [error]="error?.match"></pngx-input-text>
@@ -175,7 +175,7 @@
}
</div>
</div>
@if (formGroup.get('type').value === WorkflowTriggerType.DocumentAdded || formGroup.get('type').value === WorkflowTriggerType.DocumentUpdated || formGroup.get('type').value === WorkflowTriggerType.Scheduled) {
@if (formGroup.get('type').value === WorkflowTriggerType.DocumentAdded || formGroup.get('type').value === WorkflowTriggerType.DocumentUpdated || formGroup.get('type').value === WorkflowTriggerType.Scheduled || formGroup.get('type').value === WorkflowTriggerType.Manual) {
<div class="row mt-3">
<div class="col">
<div class="trigger-filters mb-3">
@@ -123,6 +123,10 @@ export const WORKFLOW_TYPE_OPTIONS = [
id: WorkflowTriggerType.Scheduled,
name: $localize`Scheduled`,
},
{
id: WorkflowTriggerType.Manual,
name: $localize`Manual`,
},
]
export const WORKFLOW_ACTION_OPTIONS = [
@@ -36,7 +36,7 @@
[appendTo]="appendTo"
[loading]="loading()"
[typeahead]="documentsInput$"
(mousedown)="appendTo && $event.stopImmediatePropagation()"
(mousedown)="$event.stopImmediatePropagation()"
(change)="onChange(selectedDocumentIDs)">
<ng-template ng-label-tmp let-document="item">
<div class="d-flex align-items-center">
@@ -97,7 +97,6 @@ describe('SelectComponent', () => {
let createNewVal
component.createNew.subscribe((v) => (createNewVal = v))
expect(component.allowCreateNew).toBeTruthy()
const closeSpy = jest.spyOn(component.ngSelectComponent, 'close')
component.onSearch({ term: 'foo' })
component.addItem(undefined)
expect(createNewVal).toEqual('foo')
@@ -106,8 +105,6 @@ describe('SelectComponent', () => {
component.onSearch({ term: 'baz' })
component.clickNew()
expect(createNewVal).toEqual('baz')
// dropdown should close when creating a new item so it isn't left open behind the modal
expect(closeSpy).toHaveBeenCalled()
})
it('should search items by independent normalized terms', () => {
@@ -4,7 +4,6 @@ import {
forwardRef,
Input,
Output,
ViewChild,
} from '@angular/core'
import {
FormsModule,
@@ -12,7 +11,7 @@ import {
ReactiveFormsModule,
} from '@angular/forms'
import { RouterModule } from '@angular/router'
import { NgSelectComponent, NgSelectModule } from '@ng-select/ng-select'
import { NgSelectModule } from '@ng-select/ng-select'
import { BootstrapIconComponent } from 'src/app/components/common/bootstrap-icon/bootstrap-icon.component'
import { matchesSearchText } from 'src/app/utils/text-search'
import { AbstractInputComponent } from '../abstract-input'
@@ -125,9 +124,6 @@ export class SelectComponent extends AbstractInputComponent<number | string> {
@Output()
filterDocuments = new EventEmitter<any[]>()
@ViewChild(NgSelectComponent)
ngSelectComponent: NgSelectComponent
public addItemRef: (name) => void
private _lastSearchTerm: string
@@ -154,13 +150,11 @@ export class SelectComponent extends AbstractInputComponent<number | string> {
if (name) this.createNew.next(name)
else this.createNew.next(this._lastSearchTerm)
this.clearLastSearchTerm()
this.ngSelectComponent?.close()
}
clickNew() {
this.createNew.next(this._lastSearchTerm)
this.clearLastSearchTerm()
this.ngSelectComponent?.close()
}
clearLastSearchTerm() {
@@ -49,6 +49,12 @@
<pngx-bs-icon width="1em" height="1em" name="arrow-counterclockwise" class="me-2"></pngx-bs-icon><span i18n>Reprocess</span>
</button>
@if (manualWorkflows().length > 0) {
<button ngbDropdownItem (click)="runWorkflow()" [disabled]="!userCanEdit || !userIsOwner">
<pngx-bs-icon width="1em" height="1em" name="play-circle" class="me-2"></pngx-bs-icon><span i18n>Run workflow</span>
</button>
}
<button ngbDropdownItem (click)="printDocument()" [hidden]="useNativePdfViewer || isMobile">
<pngx-bs-icon width="1em" height="1em" name="printer" class="me-2"></pngx-bs-icon><span i18n>Print</span>
</button>
@@ -1030,6 +1030,27 @@ describe('DocumentDetailComponent', () => {
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should run a workflow on the document, confirm and close modal', () => {
initNormally()
const bulkEditSpy = jest.spyOn(documentService, 'bulkEdit')
bulkEditSpy.mockReturnValue(of(true))
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
const toastSpy = jest.spyOn(toastService, 'showInfo')
component.runWorkflow()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.workflowId = 5
openModal.componentInstance.force = true
openModal.componentInstance.confirmClicked.next()
expect(bulkEditSpy).toHaveBeenCalledWith(
{ documents: [doc.id] },
'run_workflow',
{ workflow: 5, filter_documents: false }
)
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should pass remote OCR choice when reprocessing', () => {
initNormally()
const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
@@ -68,6 +68,8 @@ import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag'
import { SETTINGS_KEYS, SuggestionSource } from 'src/app/data/ui-settings'
import { User } from 'src/app/data/user'
import { Workflow } from 'src/app/data/workflow'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { CustomDatePipe } from 'src/app/pipes/custom-date.pipe'
import { DocumentTitlePipe } from 'src/app/pipes/document-title.pipe'
@@ -93,6 +95,7 @@ import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { TrashService } from 'src/app/services/trash.service'
@@ -104,6 +107,7 @@ import { DocumentDetailFieldID } from '../admin/settings/settings.component'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RunWorkflowConfirmDialogComponent } from '../common/confirm-dialog/run-workflow-confirm-dialog/run-workflow-confirm-dialog.component'
import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { DocumentTypeEditDialogComponent } from '../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
@@ -233,6 +237,7 @@ export class DocumentDetailComponent
private componentRouterService = inject(ComponentRouterService)
private deviceDetectorService = inject(DeviceDetectorService)
private savedViewService = inject(SavedViewService)
private readonly workflowService = inject(WorkflowService)
private readonly websocketStatusService = inject(WebsocketStatusService)
private readonly useNativePdfViewerSetting = this.settings.getSignal<boolean>(
SETTINGS_KEYS.USE_NATIVE_PDF_VIEWER
@@ -295,6 +300,7 @@ export class DocumentDetailComponent
readonly correspondents = signal<Correspondent[]>(undefined)
readonly documentTypes = signal<DocumentType[]>(undefined)
readonly storagePaths = signal<StoragePath[]>(undefined)
readonly manualWorkflows = signal<Workflow[]>([])
documentForm: FormGroup = new FormGroup({
title: new FormControl(''),
@@ -836,6 +842,20 @@ export class DocumentDetailComponent
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe((result) => this.users.set(result.results))
}
if (
this.permissionsService.currentUserCan(
PermissionAction.View,
PermissionType.Workflow
)
) {
this.workflowService
.list(1, 100000, null, null, {
trigger_type: WorkflowTriggerType.Manual,
enabled: true,
})
.pipe(first(), takeUntil(this.unsubscribeNotifier))
.subscribe((result) => this.manualWorkflows.set(result.results))
}
this.getCustomFields()
@@ -1559,6 +1579,39 @@ export class DocumentDetailComponent
})
}
runWorkflow() {
const modal = this.modalService.open(RunWorkflowConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Run workflow`
modal.componentInstance.message = $localize`Run a workflow on "${this.document().title}".`
modal.componentInstance.btnCaption = $localize`Run`
modal.componentInstance.workflows = this.manualWorkflows()
modal.componentInstance.confirmClicked.subscribe(() => {
modal.componentInstance.buttonsEnabled.set(false)
this.documentsService
.bulkEdit({ documents: [this.document().id] }, 'run_workflow', {
workflow: modal.componentInstance.workflowId,
filter_documents: !modal.componentInstance.force,
})
.subscribe({
next: () => {
this.toastService.showInfo(
$localize`Workflow will run on "${this.document().title}" in the background.`
)
modal.close()
},
error: (error) => {
modal.componentInstance.buttonsEnabled.set(true)
this.toastService.showError(
$localize`Error executing operation`,
error
)
},
})
})
}
private getSelectedNonLatestVersionId(): number | null {
const versions = this.document()?.versions ?? []
if (!versions.length || !this.selectedVersionId()) {
@@ -98,6 +98,11 @@
<button ngbDropdownItem (click)="mergeSelectedAsVersions()" [disabled]="!userOwnsAll || !userCanEditAll || !userCanDelete || list.allSelected || list.selectedCount < 2">
<pngx-bs-icon name="journal-bookmark-fill" class="me-2"></pngx-bs-icon><ng-container i18n>Merge as versions</ng-container>
</button>
@if (manualWorkflows.length > 0) {
<button ngbDropdownItem (click)="runWorkflowSelected()" [disabled]="!canRunWorkflow">
<pngx-bs-icon name="play-circle" class="me-2"></pngx-bs-icon><ng-container i18n>Run workflow</ng-container>
</button>
}
</div>
</div>
</div>
@@ -17,6 +17,7 @@ import { FILTER_TITLE } from 'src/app/data/filter-rule-type'
import { Results } from 'src/app/data/results'
import { StoragePath } from 'src/app/data/storage-path'
import { Tag } from 'src/app/data/tag'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
import { FilterPipe } from 'src/app/pipes/filter.pipe'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import {
@@ -36,6 +37,7 @@ import { ShareLinkBundleService } from 'src/app/services/rest/share-link-bundle.
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { TrashService } from 'src/app/services/trash.service'
@@ -83,6 +85,7 @@ describe('BulkEditorComponent', () => {
let customFieldsService: CustomFieldsService
let httpTestingController: HttpTestingController
let shareLinkBundleService: ShareLinkBundleService
let workflowService: WorkflowService
let router: Router
beforeEach(async () => {
@@ -173,6 +176,22 @@ describe('BulkEditorComponent', () => {
delete: jest.fn(),
},
},
{
provide: WorkflowService,
useValue: {
list: jest.fn(() =>
of({
results: [
{
id: 5,
name: 'Manual workflow',
triggers: [{ type: WorkflowTriggerType.Manual }],
},
],
})
),
},
},
{
provide: Router,
useValue: { navigate: jest.fn().mockResolvedValue(true) },
@@ -195,6 +214,7 @@ describe('BulkEditorComponent', () => {
customFieldsService = TestBed.inject(CustomFieldsService)
httpTestingController = TestBed.inject(HttpTestingController)
shareLinkBundleService = TestBed.inject(ShareLinkBundleService)
workflowService = TestBed.inject(WorkflowService)
router = TestBed.inject(Router)
fixture = TestBed.createComponent(BulkEditorComponent)
@@ -559,19 +579,18 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'modify_tags',
parameters: { add_tags: [101], remove_tags: [] },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should execute modify tags bulk operation for all filtered documents', () => {
@@ -608,18 +627,17 @@ describe('BulkEditorComponent', () => {
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
all: true,
filters: { title_search: 'apple' },
excluded_documents: [4],
method: 'modify_tags',
parameters: { add_tags: [101], remove_tags: [] },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
) // list reload
})
it('should execute modify tags bulk operation with confirmation dialog if enabled', () => {
@@ -646,12 +664,57 @@ describe('BulkEditorComponent', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
})
it('should load manual workflows on init', () => {
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
fixture.detectChanges()
expect(workflowService.list).toHaveBeenCalledWith(1, 100000, null, null, {
trigger_type: WorkflowTriggerType.Manual,
enabled: true,
})
expect(component.manualWorkflows).toHaveLength(1)
expect(component.manualWorkflows[0].id).toEqual(5)
})
it('should run a workflow on the selection via the run workflow dialog', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
jest.spyOn(permissionsService, 'currentUserCan').mockReturnValue(true)
jest
.spyOn(documentListViewService, 'documents', 'get')
.mockReturnValue([{ id: 3 }, { id: 4 }])
jest
.spyOn(documentListViewService, 'selected', 'get')
.mockReturnValue(new Set([3, 4]))
fixture.detectChanges()
component.runWorkflowSelected()
expect(modal).not.toBeUndefined()
expect(modal.componentInstance.workflows).toEqual(component.manualWorkflows)
modal.componentInstance.workflowId = 5
modal.componentInstance.force = true
modal.componentInstance.confirm()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
expect(req.request.body.method).toEqual('run_workflow')
expect(req.request.body.parameters).toEqual({
workflow: 5,
filter_documents: false,
})
req.flush(true)
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
) // listAllFilteredIds
})
it('should set modal dialog text accordingly for tag edit confirmation', () => {
@@ -731,19 +794,18 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'set_correspondent',
parameters: { correspondent: 101 },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should execute modify correspondent bulk operation with confirmation dialog if enabled', () => {
@@ -770,12 +832,12 @@ describe('BulkEditorComponent', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should set modal dialog text accordingly for correspondent edit confirmation', () => {
@@ -830,19 +892,18 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'set_document_type',
parameters: { document_type: 101 },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should execute modify document type bulk operation with confirmation dialog if enabled', () => {
@@ -869,12 +930,12 @@ describe('BulkEditorComponent', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should set modal dialog text accordingly for document type edit confirmation', () => {
@@ -929,19 +990,18 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'set_storage_path',
parameters: { storage_path: 101 },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should execute modify storage path bulk operation with confirmation dialog if enabled', () => {
@@ -968,12 +1028,12 @@ describe('BulkEditorComponent', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should set modal dialog text accordingly for storage path edit confirmation', () => {
@@ -1028,19 +1088,18 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'modify_custom_fields',
parameters: { add_custom_fields: [101], remove_custom_fields: [102] },
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should execute modify custom fields bulk operation with confirmation dialog if enabled', () => {
@@ -1067,12 +1126,12 @@ describe('BulkEditorComponent', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}documents/bulk_edit/`)
.flush(true)
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
// coverage for modal messages
component.setCustomFields({
@@ -1546,47 +1605,6 @@ describe('BulkEditorComponent', () => {
)
})
it.each([true, false])(
'shows queued feedback and gates the task link on permission (%s)',
(canViewTasks) => {
jest
.spyOn(permissionsService, 'currentUserCan')
.mockReturnValue(canViewTasks)
const queued = jest.spyOn(toastService, 'show')
let modal: NgbModalRef
modalService.activeInstances.subscribe((instances) => {
if (instances.length) modal = instances[0]
})
const reload = jest.spyOn(documentListViewService, 'reload')
component.setPermissions()
const close = jest.spyOn(modal, 'close')
modal.componentInstance.confirmClicked.emit({
permissions: {},
merge: true,
})
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-operation' })
expect(close).toHaveBeenCalledTimes(1)
const notification = queued.mock.calls.at(-1)[0]
expect(notification.content).toBe('Bulk operation queued.')
if (canViewTasks) {
expect(notification.actionName).toBe('View task')
notification.action()
expect(router.navigate).toHaveBeenCalledWith(['/tasks'], {
queryParams: { task_id: 'queued-operation' },
})
} else {
expect(notification.action).toBeUndefined()
expect(notification.actionName).toBeUndefined()
}
expect(reload).not.toHaveBeenCalled()
expect(documentListViewService.selected.size).toBe(0)
expect(documentListViewService.allSelected).toBe(false)
}
)
it('should support bulk permissions update', () => {
let modal: NgbModalRef
modalService.activeInstances.subscribe((m) => (modal = m[0]))
@@ -1619,9 +1637,8 @@ describe('BulkEditorComponent', () => {
let req = httpTestingController.expectOne(
`${environment.apiBaseUrl}documents/bulk_edit/`
)
req.flush({ result: 'OK', task_id: 'queued-task' })
req.flush(true)
expect(req.request.body).toEqual({
from_webui: true,
documents: [3, 4],
method: 'set_permissions',
parameters: {
@@ -1629,12 +1646,12 @@ describe('BulkEditorComponent', () => {
merge: true,
},
})
httpTestingController.expectNone(
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should not attempt to retrieve objects if user does not have permissions', () => {
@@ -1935,18 +1952,15 @@ describe('BulkEditorComponent', () => {
expect(toastServiceShowErrorSpy).toHaveBeenCalled()
expect(listReloadSpy).not.toHaveBeenCalled()
modal.componentInstance.succeeded.emit({
result: 'OK',
task_id: 'custom-fields-task',
})
expect(toastServiceShowInfoSpy).not.toHaveBeenCalled()
expect(listReloadSpy).not.toHaveBeenCalled()
httpTestingController.expectNone(
modal.componentInstance.succeeded.emit()
expect(toastServiceShowInfoSpy).toHaveBeenCalled()
expect(listReloadSpy).toHaveBeenCalled()
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`
)
httpTestingController.expectNone(
) // list reload
httpTestingController.match(
`${environment.apiBaseUrl}documents/?page=1&page_size=100000&fields=id`
)
) // listAllFilteredIds
})
it('should create share link bundle and enable manage callback', () => {
@@ -34,6 +34,7 @@ import { CustomField } from 'src/app/data/custom-field'
import { MatchingModel } from 'src/app/data/matching-model'
import { SelectionDataItem } from 'src/app/data/results'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { Workflow } from 'src/app/data/workflow'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
@@ -53,6 +54,7 @@ import {
} from 'src/app/services/rest/document.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { ShareLinkBundleService } from 'src/app/services/rest/share-link-bundle.service'
import { WorkflowService } from 'src/app/services/rest/workflow.service'
import { StoragePathService } from 'src/app/services/rest/storage-path.service'
import { TagService } from 'src/app/services/rest/tag.service'
import { SettingsService } from 'src/app/services/settings.service'
@@ -64,6 +66,7 @@ import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dial
import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
import { RunWorkflowConfirmDialogComponent } from '../../common/confirm-dialog/run-workflow-confirm-dialog/run-workflow-confirm-dialog.component'
import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
import { DocumentTypeEditDialogComponent } from '../../common/edit-dialog/document-type-edit-dialog/document-type-edit-dialog.component'
@@ -81,6 +84,7 @@ import { PermissionsDialogComponent } from '../../common/permissions-dialog/perm
import { ShareLinkBundleDialogComponent } from '../../common/share-link-bundle-dialog/share-link-bundle-dialog.component'
import { ComponentWithPermissions } from '../../with-permissions/with-permissions.component'
import { CustomFieldsBulkEditDialogComponent } from './custom-fields-bulk-edit-dialog/custom-fields-bulk-edit-dialog.component'
import { WorkflowTriggerType } from 'src/app/data/workflow-trigger'
@Component({
selector: 'pngx-bulk-editor',
@@ -104,18 +108,21 @@ export class BulkEditorComponent
private correspondentService = inject(CorrespondentService)
list = inject(DocumentListViewService)
private documentService = inject(DocumentService)
private trashService = inject(TrashService)
private modalService = inject(NgbModal)
private openDocumentService = inject(OpenDocumentsService)
private settings = inject(SettingsService)
private toastService = inject(ToastService)
private trashService = inject(TrashService)
private storagePathService = inject(StoragePathService)
private customFieldService = inject(CustomFieldsService)
public readonly permissionService = inject(PermissionsService)
private savedViewService = inject(SavedViewService)
private readonly shareLinkBundleService = inject(ShareLinkBundleService)
private readonly workflowService = inject(WorkflowService)
private readonly router = inject(Router)
public manualWorkflows: Workflow[] = []
tagSelectionModel = new FilterableDropdownSelectionModel(true)
correspondentSelectionModel = new FilterableDropdownSelectionModel()
documentTypeSelectionModel = new FilterableDropdownSelectionModel()
@@ -258,6 +265,20 @@ export class BulkEditorComponent
(result) => (this.customFieldsSelectionModel.items = result.results)
)
}
if (
this.permissionService.currentUserCan(
PermissionAction.View,
PermissionType.Workflow
)
) {
this.workflowService
.list(1, 100000, null, null, {
trigger_type: WorkflowTriggerType.Manual,
enabled: true,
})
.pipe(first())
.subscribe((result) => (this.manualWorkflows = result.results))
}
this.downloadForm
.get('downloadFileTypeArchive')
@@ -286,6 +307,28 @@ export class BulkEditorComponent
this.unsubscribeNotifier.complete()
}
get canRunWorkflow(): boolean {
return this.userCanEditAll && this.manualWorkflows.length > 0
}
runWorkflowSelected() {
const modal = this.modalService.open(RunWorkflowConfirmDialogComponent, {
backdrop: 'static',
})
modal.componentInstance.title = $localize`Run workflow`
modal.componentInstance.message = $localize`Run a workflow on ${this.getSelectionSize()} selected document(s).`
modal.componentInstance.btnCaption = $localize`Run`
modal.componentInstance.workflows = this.manualWorkflows
modal.componentInstance.confirmClicked
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(() => {
this.executeBulkEditMethod(modal, 'run_workflow', {
workflow: modal.componentInstance.workflowId,
filter_documents: !modal.componentInstance.force,
})
})
}
private executeBulkEditMethod(
modal: NgbModalRef,
method: DocumentBulkEditMethod,
@@ -299,33 +342,11 @@ export class BulkEditorComponent
.bulkEdit(overrideSelection ?? this.getSelectionQuery(), method, args)
.pipe(first())
.subscribe({
next: ({ task_id }) => this.handleOperationQueued(modal, task_id),
next: () => this.handleOperationSuccess(modal),
error: (error) => this.handleOperationError(modal, error),
})
}
private handleOperationQueued(modal: NgbModalRef, taskId: string) {
modal?.close()
this.list.selectNone()
const canViewTasks = this.permissionService.currentUserCan(
PermissionAction.View,
PermissionType.PaperlessTask
)
this.toastService.show({
content: $localize`Bulk operation queued.`,
delay: 5000,
...(canViewTasks
? {
actionName: $localize`View task`,
action: () =>
this.router.navigate(['/tasks'], {
queryParams: { task_id: taskId },
}),
}
: {}),
})
}
private executeDocumentAction(
modal: NgbModalRef,
request: Observable<any>,
@@ -1110,9 +1131,14 @@ export class BulkEditorComponent
dialog.selection = this.getSelectionQuery()
dialog.selectionCount = this.getSelectionSize()
dialog.succeeded.subscribe(({ task_id }) =>
this.handleOperationQueued(null, task_id)
)
dialog.succeeded.subscribe((result) => {
this.toastService.showSuccess($localize`Custom fields updated.`)
this.list.reload()
this.list.reduceSelectionToFilter()
this.list.selected.forEach((id) => {
this.openDocumentService.refreshDocument(id)
})
})
dialog.failed.subscribe((error) => {
this.toastService.showError(
$localize`Error updating custom fields.`,
@@ -59,7 +59,7 @@ describe('CustomFieldsBulkEditDialogComponent', () => {
it('should emit succeeded event and close modal on successful save', () => {
const editSpy = jest
.spyOn(documentService, 'bulkEdit')
.mockReturnValue(of({ result: 'OK', task_id: 'custom-fields-task' }))
.mockReturnValue(of('Success'))
const successSpy = jest.spyOn(component.succeeded, 'emit')
component.selection = [1, 2]
@@ -16,7 +16,6 @@ import { NumberComponent } from 'src/app/components/common/input/number/number.c
import { SelectComponent } from 'src/app/components/common/input/select/select.component'
import { TextComponent } from 'src/app/components/common/input/text/text.component'
import { UrlComponent } from 'src/app/components/common/input/url/url.component'
import { BulkOperationResponse } from 'src/app/data/bulk-operation'
import { CustomField, CustomFieldDataType } from 'src/app/data/custom-field'
import {
DocumentSelectionQuery,
@@ -50,7 +49,7 @@ export class CustomFieldsBulkEditDialogComponent {
CustomFieldDataType = CustomFieldDataType
@Output()
succeeded = new EventEmitter<BulkOperationResponse>()
succeeded = new EventEmitter()
@Output()
failed = new EventEmitter()
@@ -104,8 +103,6 @@ export class CustomFieldsBulkEditDialogComponent {
}
public save() {
if (this.networkActive) return
this.networkActive = true
this.documentService
.bulkEdit(this.selection, 'modify_custom_fields', {
add_custom_fields: this.form.value,
@@ -113,12 +110,11 @@ export class CustomFieldsBulkEditDialogComponent {
})
.pipe(first())
.subscribe({
next: (response) => {
next: () => {
this.activeModal.close()
this.succeeded.emit(response)
this.succeeded.emit()
},
error: (error) => {
this.networkActive = false
this.failed.emit(error)
},
})
-13
View File
@@ -1,13 +0,0 @@
import { PaperlessTaskStatus } from './paperless-task'
export interface BulkOperationResponse {
result: string
task_id: string
}
export interface BulkOperationStatus {
task_id: string
operation: string
status: PaperlessTaskStatus
error?: string
}
+1
View File
@@ -12,6 +12,7 @@ export enum WorkflowTriggerType {
DocumentAdded = 2,
DocumentUpdated = 3,
Scheduled = 4,
Manual = 5,
}
export enum ScheduleDateField {
@@ -124,42 +124,6 @@ describe('DocumentListViewService', () => {
})
})
it.each([false, true])(
'preserves the current route on a background refresh (saved view: %s)',
(savedView) => {
if (savedView) documentListViewService.activateSavedView(view)
const navigate = jest.mocked(router.navigate)
navigate.mockClear()
documentListViewService.reload(null, false)
httpTestingController
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
.flush(full_results)
expect(documentListViewService.documents).toEqual(full_results.results)
expect(navigate).not.toHaveBeenCalled()
}
)
it('preserves the route when a background refresh falls back to the first page', () => {
jest
.spyOn(documentListViewService, 'saveDocumentListView')
.mockImplementation(() => {})
documentListViewService.currentPage = 2
httpTestingController
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
.flush(full_results)
const navigate = jest.mocked(router.navigate)
navigate.mockClear()
documentListViewService.reload(null, false)
httpTestingController
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
.flush('Not found', { status: 404, statusText: 'Not found' })
httpTestingController
.expectOne((req) => req.url === `${environment.apiBaseUrl}documents/`)
.flush(full_results)
expect(documentListViewService.currentPage).toBe(1)
expect(navigate).not.toHaveBeenCalled()
})
it('should reload the list', () => {
expect(documentListViewService.currentPage).toEqual(1)
documentListViewService.reload()
@@ -380,7 +380,7 @@ export class DocumentListViewService {
queryParams: paramsFromViewState(activeListViewState),
replaceUrl: !this.router.routerState.snapshot.url.includes('?'), // in case navigating from params-less /documents
})
} else if (updateQueryParams && this._activeSavedViewId) {
} else if (this._activeSavedViewId) {
this.router.navigate([], {
queryParams: paramsFromViewState(activeListViewState, true),
queryParamsHandling: 'merge',
@@ -399,7 +399,7 @@ export class DocumentListViewService {
// this happens when applying a filter: the current page might not be available anymore due to the reduced result set.
activeListViewState.currentPage = 1
this.markChanged()
this.reload(onFinish, updateQueryParams)
this.reload()
} else if (
activeListViewState.sortField.indexOf('custom_field') === 0 &&
!this.settings
@@ -248,7 +248,6 @@ describe(`DocumentService`, () => {
)
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
from_webui: true,
documents: ids,
method,
parameters,
@@ -272,7 +271,6 @@ describe(`DocumentService`, () => {
)
expect(req.request.method).toEqual('POST')
expect(req.request.body).toEqual({
from_webui: true,
all: true,
filters: { title__icontains: 'apple' },
excluded_documents: [2, 3],
@@ -14,7 +14,6 @@ import { DocumentSuggestions } from 'src/app/data/document-suggestions'
import { FilterRule } from 'src/app/data/filter-rule'
import { Results, SelectionData } from 'src/app/data/results'
import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
import { BulkOperationResponse } from '../../data/bulk-operation'
import { queryParamsFromFilterRules } from '../../utils/query-params'
import {
PermissionAction,
@@ -39,6 +38,7 @@ export type DocumentBulkEditMethod =
| 'modify_tags'
| 'modify_custom_fields'
| 'set_permissions'
| 'run_workflow'
export interface MergeDocumentsRequest {
metadata_document_id?: number
@@ -338,15 +338,11 @@ export class DocumentService extends AbstractPaperlessService<Document> {
method: DocumentBulkEditMethod,
args: any
) {
return this.http.post<BulkOperationResponse>(
this.getResourceUrl(null, 'bulk_edit'),
{
...selection,
method: method,
from_webui: true,
parameters: args,
}
)
return this.http.post(this.getResourceUrl(null, 'bulk_edit'), {
...selection,
method: method,
parameters: args,
})
}
deleteDocuments(selection: DocumentSelectionQuery) {
@@ -438,20 +438,6 @@ describe('ConsumerStatusService', () => {
expect(updated).toBeTruthy()
})
it('forwards bulk operation completion events', () => {
const received = jest.fn()
websocketStatusService.onBulkOperationStatus().subscribe(received)
const data = {
task_id: 'bulk-task',
operation: 'set_permissions',
status: 'success',
}
websocketStatusService.connect()
server.send({ type: WebsocketStatusType.BULK_OPERATION_STATUS, data })
websocketStatusService.disconnect()
expect(received).toHaveBeenCalledWith(data)
})
it('should ignore keep-alive heartbeat messages from the server', () => {
let updated = false
let deleted = false
@@ -1,7 +1,6 @@
import { Injectable, inject, signal } from '@angular/core'
import { Subject } from 'rxjs'
import { environment } from 'src/environments/environment'
import { BulkOperationStatus } from '../data/bulk-operation'
import { User } from '../data/user'
import { WebsocketDocumentUpdatedMessage } from '../data/websocket-document-updated-message'
import { WebsocketDocumentsDeletedMessage } from '../data/websocket-documents-deleted-message'
@@ -13,7 +12,6 @@ export enum WebsocketStatusType {
DOCUMENTS_DELETED = 'documents_deleted',
DOCUMENT_UPDATED = 'document_updated',
HEARTBEAT = 'heartbeat',
BULK_OPERATION_STATUS = 'bulk_operation_status',
}
// see ProgressStatusOptions in src/documents/plugins/helpers.py
@@ -118,13 +116,6 @@ export class WebsocketStatusService {
private readonly documentDeletedSubject = new Subject<boolean>()
private readonly documentUpdatedSubject =
new Subject<WebsocketDocumentUpdatedMessage>()
private readonly bulkOperationStatusSubject =
new Subject<BulkOperationStatus>()
onBulkOperationStatus() {
return this.bulkOperationStatusSubject.asObservable()
}
private readonly connectionStatusSubject = new Subject<boolean>()
private get(taskId: string, filename?: string) {
@@ -201,7 +192,6 @@ export class WebsocketStatusService {
| WebsocketProgressMessage
| WebsocketDocumentsDeletedMessage
| WebsocketDocumentUpdatedMessage
| BulkOperationStatus
} = JSON.parse(ev.data)
switch (type) {
@@ -219,12 +209,6 @@ export class WebsocketStatusService {
this.handleProgressUpdate(messageData as WebsocketProgressMessage)
break
case WebsocketStatusType.BULK_OPERATION_STATUS:
this.bulkOperationStatusSubject.next(
messageData as BulkOperationStatus
)
break
case WebsocketStatusType.HEARTBEAT:
// keep-alive from the server, see paperless.consumers.StatusConsumer
break
+56 -53
View File
@@ -42,13 +42,13 @@ from documents.plugins.helpers import DocumentsStatusManager
from documents.tasks import bulk_update_documents
from documents.tasks import consume_file
from documents.tasks import remove_document_from_index
from documents.tasks import run_workflows_for_documents
from documents.tasks import update_document_content_maybe_archive_file
from documents.versioning import get_latest_version_for_root
from documents.versioning import get_root_document
from documents.versioning import versions_newest_first
if TYPE_CHECKING:
from collections.abc import Callable
from collections.abc import Mapping
from django.contrib.auth.models import User
@@ -132,24 +132,9 @@ def _scratch_path(name: str) -> Path:
return Path(tempfile.mkdtemp(dir=settings.SCRATCH_DIR)) / name
def _finish_metadata_edit(
document_ids: list[int],
update_callback: Callable[[list[int]], None] | None,
) -> None:
if update_callback is not None:
update_callback(document_ids)
else:
bulk_update_documents.apply_async(
kwargs={"document_ids": document_ids},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
def set_correspondent(
doc_ids: list[int],
correspondent: Correspondent,
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
if correspondent:
correspondent = Correspondent.objects.only("pk").get(id=correspondent)
@@ -162,17 +147,15 @@ def set_correspondent(
affected_docs = list(qs.values_list("pk", flat=True))
qs.update(correspondent=correspondent)
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
def set_storage_path(
doc_ids: list[int],
storage_path: StoragePath,
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
def set_storage_path(doc_ids: list[int], storage_path: StoragePath) -> Literal["OK"]:
if storage_path:
storage_path = StoragePath.objects.only("pk").get(id=storage_path)
@@ -186,17 +169,15 @@ def set_storage_path(
affected_docs = list(qs.values_list("pk", flat=True))
qs.update(storage_path=storage_path)
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
def set_document_type(
doc_ids: list[int],
document_type: DocumentType,
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
def set_document_type(doc_ids: list[int], document_type: DocumentType) -> Literal["OK"]:
if document_type:
document_type = DocumentType.objects.only("pk").get(id=document_type)
@@ -208,17 +189,15 @@ def set_document_type(
affected_docs = list(qs.values_list("pk", flat=True))
qs.update(document_type=document_type)
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
def add_tag(
doc_ids: list[int],
tag: int,
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
def add_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
tag_obj = Tag.objects.get(pk=tag)
tags_to_add = [tag_obj, *tag_obj.get_ancestors()]
@@ -239,17 +218,15 @@ def add_tag(
DocumentTagRelationship.objects.bulk_create(to_create)
if affected_docs:
_finish_metadata_edit(list(affected_docs), update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": list(affected_docs)},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
def remove_tag(
doc_ids: list[int],
tag: int,
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
def remove_tag(doc_ids: list[int], tag: int) -> Literal["OK"]:
tag_obj = Tag.objects.get(pk=tag)
tag_ids = [tag_obj.id, *tag_obj.get_descendants_pks()]
@@ -262,7 +239,10 @@ def remove_tag(
qs.delete()
if affected_docs:
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
@@ -271,8 +251,6 @@ def modify_tags(
doc_ids: list[int],
add_tags: list[int],
remove_tags: list[int],
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
qs = Document.objects.filter(id__in=doc_ids).only("pk")
affected_docs = list(qs.values_list("pk", flat=True))
@@ -316,7 +294,10 @@ def modify_tags(
)
if affected_docs:
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
@@ -325,8 +306,6 @@ def modify_custom_fields(
doc_ids: list[int],
add_custom_fields: list[int] | dict,
remove_custom_fields: list[int],
*,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
qs = Document.objects.filter(id__in=doc_ids).only("pk")
affected_docs = list(qs.values_list("pk", flat=True))
@@ -389,7 +368,10 @@ def modify_custom_fields(
field_id__in=remove_custom_fields,
).hard_delete()
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
@@ -464,13 +446,31 @@ def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
return "OK"
def run_workflow(
doc_ids: list[int],
workflow: int,
*,
filter_documents: bool = True,
) -> Literal["OK"]:
"""
Manually run a single workflow against the given documents.
Done asynchronously via a Celery task.
"""
run_workflows_for_documents.delay(
document_ids=doc_ids,
workflow_id=workflow,
filter_documents=filter_documents,
)
return "OK"
def set_permissions(
doc_ids: list[int],
set_permissions: PermissionsSpec,
*,
owner: User | None = None,
merge: bool = False,
update_callback: Callable[[list[int]], None] | None = None,
) -> Literal["OK"]:
qs = Document.objects.filter(id__in=doc_ids).select_related("owner")
@@ -488,7 +488,10 @@ def set_permissions(
merge=merge,
)
_finish_metadata_edit(affected_docs, update_callback)
bulk_update_documents.apply_async(
kwargs={"document_ids": affected_docs},
headers={"trigger_source": PaperlessTask.TriggerSource.SYSTEM},
)
return "OK"
-107
View File
@@ -1,107 +0,0 @@
"""Queued document metadata edits."""
from typing import Any
from auditlog.context import set_actor
from celery import shared_task
from django.conf import settings
from django.contrib.auth.models import User
from django.db import transaction
from rest_framework.exceptions import PermissionDenied
from documents import bulk_edit
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.permissions import has_document_operation_permissions
from documents.tasks import bulk_update_documents
MODIFIED_FIELDS = {
"set_correspondent": "correspondent",
"set_document_type": "document_type",
"set_storage_path": "storage_path",
"add_tag": "tags",
"remove_tag": "tags",
"modify_tags": "tags",
"modify_custom_fields": "custom_fields",
"set_permissions": None,
}
QUEUED_OPERATIONS = frozenset(MODIFIED_FIELDS)
def snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
"""
Returns each document's current value of field, for the audit log.
Tags and custom fields are one row per value, so they are gathered
into a sorted list of pks per document (empty when there are none).
Reading them through Document.values() instead would join those rows
and return one arbitrary value per document.
"""
if field == "tags":
rows = (
Document.tags.through.objects.filter(document_id__in=doc_ids)
.order_by("tag_id")
.values_list("document_id", "tag_id")
)
elif field == "custom_fields":
rows = (
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
.order_by("pk")
.values_list("document_id", "pk")
)
else:
return dict(
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
)
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
for doc_id, pk in rows:
values[doc_id].append(pk)
return values
@shared_task
def execute_bulk_operation(
operation: str,
document_ids: list[int],
parameters: dict,
user_id: int,
) -> dict:
if operation not in QUEUED_OPERATIONS:
raise ValueError(f"Unsupported bulk operation: {operation}")
method = getattr(bulk_edit, operation)
user = User.objects.get(pk=user_id)
if not has_document_operation_permissions(
user=user,
documents=document_ids,
method=method,
parameters=parameters,
):
raise PermissionDenied("Insufficient permissions to execute bulk operation")
affected_ids: list[int] = []
modified_field = MODIFIED_FIELDS[operation]
# Keep metadata writes and their audit entries together. Follow-up processing
# runs after commit, within this same tracked task.
with set_actor(user), transaction.atomic():
old_values = (
snapshot_field(document_ids, modified_field)
if settings.AUDIT_LOG_ENABLED and modified_field
else {}
)
method(document_ids, **parameters, update_callback=affected_ids.extend)
if settings.AUDIT_LOG_ENABLED and modified_field:
from auditlog.models import LogEntry
new_values = snapshot_field(document_ids, modified_field)
for doc in Document.objects.filter(pk__in=document_ids):
LogEntry.objects.log_create(
instance=doc,
changes={modified_field: [old_values[doc.pk], new_values[doc.pk]]},
action=LogEntry.Action.UPDATE,
actor=user,
additional_data={"reason": f"Bulk edit: {operation}"},
)
if affected_ids:
bulk_update_documents(list(dict.fromkeys(affected_ids)))
return {"operation": operation, "document_count": len(document_ids)}
+6 -2
View File
@@ -631,7 +631,9 @@ class ConsumerPlugin(
pk=self.metadata.actor_id,
).first()
if actor is not None:
from auditlog.context import set_actor # type: ignore[import-untyped]
from auditlog.context import ( # type: ignore[import-untyped]
set_actor,
)
with set_actor(actor):
original_document.save()
@@ -649,7 +651,9 @@ class ConsumerPlugin(
# Create a log entry for the version addition, if enabled
if settings.AUDIT_LOG_ENABLED:
from auditlog.models import LogEntry # type: ignore[import-untyped]
from auditlog.models import ( # type: ignore[import-untyped]
LogEntry,
)
LogEntry.objects.log_create(
instance=root_doc,
+13
View File
@@ -35,6 +35,7 @@ from django_filters.rest_framework import DateTimeFilter
from django_filters.rest_framework import Filter
from django_filters.rest_framework import FilterSet
from django_filters.rest_framework import MultipleChoiceFilter
from django_filters.rest_framework import NumberFilter
from drf_spectacular.utils import extend_schema_field
from guardian.utils import get_group_obj_perms_model
from guardian.utils import get_user_obj_perms_model
@@ -52,6 +53,7 @@ from documents.models import ShareLink
from documents.models import ShareLinkBundle
from documents.models import StoragePath
from documents.models import Tag
from documents.models import Workflow
from documents.permissions import annotate_authorizing_fields
from documents.permissions import permitted_document_ids
from documents.permissions import permitted_object_ids
@@ -249,6 +251,17 @@ class CustomFieldFilterSet(FilterSet):
}
class WorkflowFilterSet(FilterSet):
trigger_type = NumberFilter(field_name="triggers__type", distinct=True)
class Meta:
model = Workflow
fields = {
"id": ID_KWARGS,
"enabled": ["exact"],
}
@extend_schema_field(serializers.CharField)
class CustomFieldsFilter(Filter):
def filter(self, qs, value):
+1
View File
@@ -684,6 +684,7 @@ def document_matches_workflow(
trigger_type == WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED
or trigger_type == WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED
or trigger_type == WorkflowTrigger.WorkflowTriggerType.SCHEDULED
or trigger_type == WorkflowTrigger.WorkflowTriggerType.MANUAL
):
trigger_matched, reason = existing_document_matches_workflow(
document,
@@ -0,0 +1,43 @@
# Generated by Django 5.2.18 on 2026-11-06 17:12
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0028_replace_tag_tree_fields"),
]
operations = [
migrations.AlterField(
model_name="workflowrun",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Consumption Started"),
(2, "Document Added"),
(3, "Document Updated"),
(4, "Scheduled"),
(5, "Manual"),
],
null=True,
verbose_name="workflow trigger type",
),
),
migrations.AlterField(
model_name="workflowtrigger",
name="type",
field=models.PositiveSmallIntegerField(
choices=[
(1, "Consumption Started"),
(2, "Document Added"),
(3, "Document Updated"),
(4, "Scheduled"),
(5, "Manual"),
],
default=1,
verbose_name="Workflow Trigger Type",
),
),
]
+1
View File
@@ -1532,6 +1532,7 @@ class WorkflowTrigger(models.Model):
DOCUMENT_ADDED = 2, _("Document Added")
DOCUMENT_UPDATED = 3, _("Document Updated")
SCHEDULED = 4, _("Scheduled")
MANUAL = 5, _("Manual")
class DocumentSourceChoices(models.IntegerChoices):
CONSUME_FOLDER = DocumentSource.ConsumeFolder.value, _("Consume Folder")
+6 -112
View File
@@ -31,7 +31,6 @@ from guardian.core import ObjectPermissionChecker
from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission
from rest_framework.permissions import BasePermission
from rest_framework.permissions import DjangoModelPermissions
from rest_framework.permissions import DjangoObjectPermissions
from documents.models import Document
@@ -64,17 +63,6 @@ class PermissionGranteesUpdate(TypedDict, total=False):
PermissionsSpec: TypeAlias = Mapping[PermissionAction, PermissionGranteesUpdate]
class PaperlessModelPermissions(DjangoModelPermissions):
"""Require basic global model permissions."""
perms_map = {
**DjangoModelPermissions.perms_map,
"GET": ["%(app_label)s.view_%(model_name)s"],
"OPTIONS": ["%(app_label)s.view_%(model_name)s"],
"HEAD": ["%(app_label)s.view_%(model_name)s"],
}
class PaperlessObjectPermissions(DjangoObjectPermissions):
"""
A permissions backend that checks for object-level permissions
@@ -588,9 +576,12 @@ def annotate_document_count_by_ids(
or CustomFieldInstance)
related_object_field: field on the relation pointing back to queryset pk
document_ids: the document ids to count against -- a concrete list/set,
or a simple (already resolved) queryset of ids. Keep
the list small: it is embedded in the statement as
literal parameters.
or a simple (already resolved) queryset of ids. Callers
that need this filtered by a complex condition (e.g. a
permission check) should resolve it to a concrete list
first if the same ids will be reused across multiple
calls, rather than passing the complex queryset itself
into each -- see ``_get_selection_data_for_queryset``.
target_field: field on the relation pointing to Document id
"""
@@ -739,100 +730,3 @@ class AcknowledgeTasksPermissions(GlobalPermissionsByMethod):
"OPTIONS": ["documents.change_paperlesstask"],
"POST": ["documents.change_paperlesstask"],
}
def has_document_operation_permissions(
*,
user: User,
documents: list[int],
method,
parameters: dict[str, Any],
) -> bool:
from documents import bulk_edit
if not user.is_active:
return False
if user.is_superuser:
return True
root_docs = {
get_root_document(doc)
for doc in Document.objects.select_related(
"owner",
"root_document__owner",
).filter(pk__in=documents)
}
user_is_owner_of_all_documents = all(
(doc.owner == user or doc.owner is None) for doc in root_docs
)
# check global and object permissions for all documents
has_perms = (
user.has_perm(
"documents.change_document",
)
and not Document.global_objects.filter(pk__in=documents)
.exclude(pk__in=permitted_document_ids(user, perm="change_document"))
.exists()
)
# check ownership for methods that change original document
if (
(
has_perms
and method
in [
bulk_edit.set_permissions,
bulk_edit.delete,
bulk_edit.rotate,
bulk_edit.delete_pages,
bulk_edit.edit_pdf,
bulk_edit.merge_as_versions,
bulk_edit.remove_password,
]
)
or (
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
):
has_perms = has_perms and user_is_owner_of_all_documents
# check global add permissions for methods that create documents
if (
has_perms
and (
method in [bulk_edit.split, bulk_edit.merge]
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.add_document")
):
has_perms = False
# check global delete permissions for methods that delete documents
if (
has_perms
and (
method == bulk_edit.delete
# Sources stop being documents of their own, and removing one
# again afterwards needs delete_document
or method == bulk_edit.merge_as_versions
or (
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and parameters.get("delete_original")
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.delete_document")
):
has_perms = False
return has_perms
+34
View File
@@ -1829,6 +1829,7 @@ class BulkEditSerializer(
"modify_tags",
"modify_custom_fields",
"set_permissions",
"run_workflow",
*LEGACY_DOCUMENT_ACTION_METHODS,
],
label="Method",
@@ -1918,6 +1919,8 @@ class BulkEditSerializer(
return bulk_edit.reprocess
elif method == "set_permissions":
return bulk_edit.set_permissions
elif method == "run_workflow":
return bulk_edit.run_workflow
elif method == "rotate":
return bulk_edit.rotate
elif method == "merge":
@@ -2157,6 +2160,35 @@ class BulkEditSerializer(
else:
parameters["remote_ocr"] = False
def _validate_parameters_run_workflow(self, parameters) -> None:
if "workflow" not in parameters:
raise serializers.ValidationError("workflow not specified")
try:
workflow = Workflow.objects.get(id=parameters["workflow"])
except (Workflow.DoesNotExist, ValueError, TypeError):
raise serializers.ValidationError("Workflow does not exist")
request = self.context.get("request")
user = getattr(request, "user", None)
if user is not None and not user.has_perm("documents.view_workflow"):
raise PermissionDenied(
"Insufficient permissions to run this workflow",
)
if not workflow.enabled:
raise serializers.ValidationError("Workflow is not enabled")
if not workflow.triggers.filter(
type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
).exists():
raise serializers.ValidationError(
"Workflow does not have a manual trigger",
)
if "filter_documents" in parameters:
if not isinstance(parameters["filter_documents"], bool):
raise serializers.ValidationError(
"filter_documents must be a boolean",
)
else:
parameters["filter_documents"] = True
def validate_parameters_remove_password(self, parameters):
if "password" not in parameters:
raise serializers.ValidationError("password not specified")
@@ -2223,6 +2255,8 @@ class BulkEditSerializer(
self.validate_parameters_remove_password(parameters)
elif method == bulk_edit.reprocess:
self._validate_parameters_reprocess(parameters)
elif method == bulk_edit.run_workflow:
self._validate_parameters_run_workflow(parameters)
return attrs
+17 -49
View File
@@ -8,7 +8,6 @@ from pathlib import Path
from typing import TYPE_CHECKING
from typing import Any
from asgiref.sync import async_to_sync
from celery import shared_task
from celery.signals import before_task_publish
from celery.signals import task_failure
@@ -17,7 +16,6 @@ from celery.signals import task_prerun
from celery.signals import task_revoked
from celery.signals import worker_process_init
from celery.signals import worker_process_shutdown
from channels.layers import get_channel_layer
from django.conf import settings
from django.contrib.auth.models import Group
from django.contrib.auth.models import User
@@ -864,6 +862,8 @@ def run_workflows(
logging_group: uuid.UUID | None = None,
overrides: DocumentMetadataOverrides | None = None,
original_file: Path | None = None,
*,
skip_matching: bool = False,
) -> tuple[DocumentMetadataOverrides, str] | None:
"""
Execute workflows matching a document for the given trigger. When `overrides` is provided
@@ -874,6 +874,10 @@ def run_workflows(
`document.source_path` (Document) or `document.original_file` (ConsumableDocument).
Passing `workflow_to_run` skips the workflow query (currently only used by scheduled runs).
Passing `skip_matching` runs the workflow's actions without evaluating its trigger
filters against the document. It is only meaningful together with `workflow_to_run`
(e.g. a manual run the user has explicitly forced onto the selected documents).
"""
use_overrides = overrides is not None
@@ -923,7 +927,11 @@ def run_workflows(
)
break
if matching.document_matches_workflow(document, workflow, trigger_type):
if skip_matching or matching.document_matches_workflow(
document,
workflow,
trigger_type,
):
action: WorkflowAction
has_move_to_trash_action = False
for action in workflow.actions.order_by("order", "pk"):
@@ -1053,8 +1061,6 @@ def run_workflows(
# Task tracking -- Celery signal handlers
# ---------------------------------------------------------------------------
BULK_OPERATION_TASK_NAME = "documents.bulk_operations.execute_bulk_operation"
TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
"documents.tasks.consume_file": PaperlessTask.TaskType.CONSUME_FILE,
"documents.tasks.train_classifier": PaperlessTask.TaskType.TRAIN_CLASSIFIER,
@@ -1068,7 +1074,6 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
"documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
"documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
"documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
BULK_OPERATION_TASK_NAME: PaperlessTask.TaskType.BULK_UPDATE,
}
_CELERY_STATE_TO_STATUS: dict[str, PaperlessTask.Status] = {
@@ -1086,7 +1091,8 @@ def _extract_input_data(
For consume_file tasks this includes the filename, MIME type, and any
non-null overrides from the DocumentMetadataOverrides object. For
mail_fetch tasks it captures the account_ids list.
mail_fetch tasks it captures the account_ids list. All other task
types store no input data and return {}.
"""
if task_type == PaperlessTask.TaskType.CONSUME_FILE:
input_doc = task_kwargs.get("input_doc")
@@ -1151,7 +1157,7 @@ def _extract_owner_id(
task_type: PaperlessTask.TaskType,
task_kwargs: dict,
) -> int | None:
"""Return the document owner for consumption tasks."""
"""Return the owner_id from consume_file overrides, or None for all other task types."""
if task_type != PaperlessTask.TaskType.CONSUME_FILE:
return None
overrides = task_kwargs.get("overrides")
@@ -1190,16 +1196,9 @@ def before_task_publish_handler(
_, task_kwargs, _ = body
task_id = headers["id"]
if task_name == BULK_OPERATION_TASK_NAME:
input_data = {
"operation": task_kwargs["operation"],
"document_count": len(task_kwargs["document_ids"]),
}
owner_id = task_kwargs["user_id"]
else:
input_data = _extract_input_data(task_type, task_kwargs)
owner_id = _extract_owner_id(task_type, task_kwargs)
input_data = _extract_input_data(task_type, task_kwargs)
trigger_source = _determine_trigger_source(headers)
owner_id = _extract_owner_id(task_type, task_kwargs)
# A retried task is republished with the same task_id, so this fires
# again for it; get_or_create keeps the original PENDING record
@@ -1239,30 +1238,6 @@ def task_prerun_handler(sender=None, task_id=None, task=None, **kwargs) -> None:
logger.exception("Setting PaperlessTask started failed")
def _send_bulk_operation_status(task: PaperlessTask | None) -> None:
"""Send the recorded outcome to the initiator."""
try:
if task is None or task.owner_id is None:
return
result_data = task.result_data or {}
async_to_sync(get_channel_layer().group_send)(
f"bulk_operations_{task.owner_id}",
{
"type": "bulk_operation_status",
"data": {
"task_id": task.task_id,
"status": task.status,
"operation": task.input_data["operation"],
"error": result_data.get("error_message")
or result_data.get("reason", ""),
},
},
)
except Exception:
# Just a notification failure
logger.exception("Sending bulk operation status failed")
@task_postrun.connect
def task_postrun_handler(
sender=None,
@@ -1320,8 +1295,6 @@ def task_postrun_handler(
changed_fields.append("status")
task_instance.save(update_fields=changed_fields)
if task and task.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_instance)
except Exception: # pragma: no cover
logger.exception("Updating PaperlessTask failed")
@@ -1387,8 +1360,6 @@ def task_failure_handler(
date_started - date_created
).total_seconds()
task_qs.update(**update_fields)
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
logger.exception("Updating PaperlessTask on failure failed")
@@ -1425,16 +1396,13 @@ def task_revoked_handler(
reason = "Task expired before execution."
elif terminated:
reason = "Task terminated during execution."
task_qs = PaperlessTask.objects.filter(task_id=task_id)
task_qs.update(
PaperlessTask.objects.filter(task_id=task_id).update(
status=PaperlessTask.Status.REVOKED,
date_done=timezone.now(),
result_data={
"reason": (reason + " Check worker logs for details."),
},
)
if sender and sender.name == BULK_OPERATION_TASK_NAME:
_send_bulk_operation_status(task_qs.first())
except Exception: # pragma: no cover
logger.exception("Updating PaperlessTask on revocation failed")
+46 -15
View File
@@ -12,7 +12,6 @@ from celery import Task
from celery import shared_task
from django.conf import settings
from django.contrib.contenttypes.models import ContentType
from django.contrib.sessions.models import Session
from django.db import models
from django.db import transaction
from django.db.models.signals import post_save
@@ -53,6 +52,7 @@ from documents.models import ShareLink
from documents.models import ShareLinkBundle
from documents.models import StoragePath
from documents.models import Tag
from documents.models import Workflow
from documents.models import WorkflowRun
from documents.models import WorkflowTrigger
from documents.plugins.base import ConsumeTaskPlugin
@@ -691,6 +691,51 @@ def check_scheduled_workflows() -> None:
)
@shared_task
def run_workflows_for_documents(
document_ids: list[int],
workflow_id: int,
*,
filter_documents: bool = True,
) -> None:
"""
Run a single workflow manually against the given documents.
``filter_documents`` controls whether the workflow's trigger filters are applied.
When True, only documents matching the existing filters will have the workflow run.
When False, the workflow will be run on all specified documents regardless of the filters.
``
"""
try:
workflow = Workflow.objects.get(id=workflow_id)
except Workflow.DoesNotExist:
logger.warning(
f"Manual workflow run requested for missing workflow {workflow_id}",
)
return
for document in Document.objects.filter(
id__in=document_ids,
root_document__isnull=True,
):
try:
run_workflows(
trigger_type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
document=document,
workflow_to_run=workflow,
skip_matching=not filter_documents,
)
# Manual runs dont send document_updated signal, so ws here to ensure clients are refreshed
send_websocket_document_updated(
sender=None,
document=document,
)
except Exception:
logger.exception(
f"Error running workflow {workflow} on document {document.pk}",
)
def update_document_parent_tags(tag: Tag, new_parent: Tag) -> None:
"""
When a tag's parent changes, ensure all documents containing the tag also have
@@ -926,17 +971,3 @@ def cleanup_expired_share_link_bundles() -> None:
)
if count:
logger.info("Deleted %s expired share link bundle(s)", count)
@shared_task
def clear_expired_sessions() -> None:
"""
Remove expired database sessions in batches to limit the size of each delete.
"""
expired = Session.objects.filter(expire_date__lt=timezone.now())
while session_keys := list(
expired.order_by("expire_date").values_list("pk", flat=True)[:500],
):
expired.filter(pk__in=session_keys).delete()
logger.info("Cleared expired sessions")
@@ -15,7 +15,6 @@ from paperless.models import ColorConvertChoices
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import UserFactory
from paperless_testing.http import read_streaming_response
from paperless_testing.permissions import grant_global
class TestApiAppConfig(DirectoriesMixin, APITestCase):
@@ -27,67 +26,6 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
user = UserFactory(username="temp_admin", superuser=True)
self.client.force_authenticate(user=user)
def test_api_read_config_requires_view_permission(self) -> None:
for permissions in (
(),
("change_applicationconfiguration",),
("view_document",),
):
user = UserFactory()
grant_global(user, *permissions)
self.client.force_authenticate(user=user)
for endpoint in (self.ENDPOINT, f"{self.ENDPOINT}1/"):
for method in ("get", "head", "options"):
with self.subTest(
permissions=permissions,
endpoint=endpoint,
method=method,
):
response = getattr(self.client, method)(endpoint)
self.assertEqual(
response.status_code,
status.HTTP_403_FORBIDDEN,
)
def test_api_read_config_with_view_permission(self) -> None:
user = UserFactory()
grant_global(user, "view_applicationconfiguration")
self.client.force_authenticate(user=user)
for endpoint in (self.ENDPOINT, f"{self.ENDPOINT}1/"):
for method in ("get", "head", "options"):
with self.subTest(endpoint=endpoint, method=method):
response = getattr(self.client, method)(endpoint)
self.assertEqual(response.status_code, status.HTTP_200_OK)
def test_api_view_config_permission_does_not_allow_writes(self) -> None:
user = UserFactory()
grant_global(user, "view_applicationconfiguration")
self.client.force_authenticate(user=user)
for method in ("put", "patch", "delete"):
with self.subTest(method=method):
response = getattr(self.client, method)(f"{self.ENDPOINT}1/")
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
def test_api_change_config_permission_allows_updates(self) -> None:
user = UserFactory()
grant_global(user, "change_applicationconfiguration")
self.client.force_authenticate(user=user)
response = self.client.patch(
f"{self.ENDPOINT}1/",
{"app_title": "Updated title"},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(
ApplicationConfiguration.objects.get().app_title,
"Updated title",
)
def test_api_get_config(self) -> None:
"""
GIVEN:
@@ -287,28 +225,6 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
self.assertEqual(config.user_args, user_args)
self.assertEqual(config.barcode_tag_mapping, barcode_tag_mapping)
def test_api_update_config_rejects_reserved_user_args(self) -> None:
"""
GIVEN:
- API request to update app config with user_args which set an
argument the parser manages itself
WHEN:
- API is called
THEN:
- Request is rejected with a 400 naming the argument
- Config is not updated
"""
response = self.client.patch(
f"{self.ENDPOINT}1/",
json.dumps({"user_args": json.dumps({"sidecar": "/tmp/x.py"})}),
content_type="application/json",
)
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
self.assertIn("sidecar", str(response.data["user_args"]))
config = ApplicationConfiguration.objects.first()
assert config is not None
self.assertIsNone(config.user_args)
def test_api_update_config_rejects_invalid_json_objects(self) -> None:
"""
GIVEN:
+122 -87
View File
@@ -7,7 +7,6 @@ from django.test import override_settings
from rest_framework import status
from rest_framework.test import APITestCase
from documents.bulk_operations import execute_bulk_operation
from documents.models import Correspondent
from documents.models import CustomField
from documents.models import CustomFieldInstance
@@ -15,7 +14,11 @@ from documents.models import Document
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from documents.models import Workflow
from documents.models import WorkflowAction
from documents.models import WorkflowTrigger
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import UserFactory
from paperless_testing.permissions import grant_all_global
from paperless_testing.permissions import grant_global
@@ -30,18 +33,9 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.user = user
self.client.force_authenticate(user=user)
patcher = mock.patch("documents.bulk_operations.bulk_update_documents")
patcher = mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
self.async_task = patcher.start()
self.addCleanup(patcher.stop)
# Exercise accepted requests and worker execution together in these
# operation/validation tests. Queue-only behavior is tested separately.
enqueue = mock.patch(
"documents.views.execute_bulk_operation.apply_async",
side_effect=lambda *, kwargs, headers: self.run_operation(kwargs),
)
enqueue.start()
self.addCleanup(enqueue.stop)
self.c1 = Correspondent.objects.create(name="c1")
self.c2 = Correspondent.objects.create(name="c2")
self.dt1 = DocumentType.objects.create(name="dt1")
@@ -71,16 +65,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.cf1 = CustomField.objects.create(name="cf1", data_type="string")
self.cf2 = CustomField.objects.create(name="cf2", data_type="string")
@staticmethod
def run_operation(kwargs):
execute_bulk_operation(**kwargs)
return mock.Mock(id="test-bulk-operation")
def setup_mock(self, m, method_name, return_value="OK") -> None:
m.return_value = return_value
m.__name__ = method_name
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_set_correspondent(self, bulk_update_task_mock) -> None:
self.assertNotEqual(self.doc1.correspondent, self.c1)
response = self.client.post(
@@ -99,11 +88,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(self.doc1.correspondent, self.c1)
bulk_update_task_mock.assert_called_once()
self.assertCountEqual(
bulk_update_task_mock.call_args.args[0],
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
[self.doc1.pk],
)
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_unset_correspondent(self, bulk_update_task_mock) -> None:
self.doc1.correspondent = self.c1
self.doc1.save()
@@ -125,7 +114,7 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.doc1.refresh_from_db()
self.assertIsNone(self.doc1.correspondent)
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_set_type(self, bulk_update_task_mock) -> None:
self.assertNotEqual(self.doc1.document_type, self.dt1)
response = self.client.post(
@@ -144,11 +133,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(self.doc1.document_type, self.dt1)
bulk_update_task_mock.assert_called_once()
self.assertCountEqual(
bulk_update_task_mock.call_args.args[0],
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
[self.doc1.pk],
)
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_unset_type(self, bulk_update_task_mock) -> None:
self.doc1.document_type = self.dt1
self.doc1.save()
@@ -169,11 +158,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertIsNone(self.doc1.document_type)
bulk_update_task_mock.assert_called_once()
self.assertCountEqual(
bulk_update_task_mock.call_args.args[0],
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
[self.doc1.pk],
)
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_add_tag(self, bulk_update_task_mock) -> None:
self.assertFalse(self.doc1.tags.filter(pk=self.t1.pk).exists())
@@ -195,11 +184,11 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
bulk_update_task_mock.assert_called_once()
self.assertCountEqual(
bulk_update_task_mock.call_args.args[0],
bulk_update_task_mock.call_args.kwargs["kwargs"]["document_ids"],
[self.doc1.pk],
)
@mock.patch("documents.bulk_operations.bulk_update_documents")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_api_remove_tag(self, bulk_update_task_mock) -> None:
self.doc1.tags.add(self.t1)
@@ -238,9 +227,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
self.assertEqual(kwargs["add_tags"], [self.t1.id])
self.assertEqual(kwargs["remove_tags"], [self.t2.id])
@@ -295,9 +281,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
self.assertEqual(kwargs["add_custom_fields"], [self.cf1.id])
self.assertEqual(kwargs["remove_custom_fields"], [self.cf2.id])
@@ -366,9 +349,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertListEqual(args[0], [self.doc1.id, self.doc3.id])
self.assertEqual(kwargs["add_custom_fields"], {self.cf1.id: "foo"})
self.assertEqual(kwargs["remove_custom_fields"], [self.cf2.id])
@@ -530,9 +510,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(len(kwargs), 0)
@@ -547,9 +524,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(len(kwargs), 0)
@@ -574,9 +548,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(len(kwargs), 0)
@@ -591,9 +562,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"remote_ocr": False})
@@ -616,12 +584,114 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"remote_ocr": True})
def _make_manual_workflow(self) -> Workflow:
trigger = WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
)
action = WorkflowAction.objects.create()
workflow = Workflow.objects.create(name="Manual workflow", order=0)
workflow.triggers.add(trigger)
workflow.actions.add(action)
return workflow
def _run_workflow(self, parameters, documents=None):
return self.client.post(
"/api/documents/bulk_edit/",
json.dumps(
{
"documents": documents or [self.doc1.id],
"method": "run_workflow",
"parameters": parameters,
},
),
content_type="application/json",
)
@mock.patch("documents.views.bulk_edit.run_workflow")
def test_run_workflow(self, m) -> None:
"""A valid run_workflow call passes the docs, workflow id and filter flag through."""
self.setup_mock(m, "run_workflow")
workflow = self._make_manual_workflow()
response = self._run_workflow({"workflow": workflow.id})
self.assertEqual(response.status_code, status.HTTP_200_OK)
args, kwargs = m.call_args
self.assertEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs, {"workflow": workflow.id, "filter_documents": True})
self._run_workflow({"workflow": workflow.id, "filter_documents": False})
self.assertFalse(m.call_args.kwargs["filter_documents"])
@mock.patch("documents.views.bulk_edit.run_workflow")
def test_run_workflow_invalid_parameters(self, m) -> None:
"""Invalid run_workflow parameters are rejected before dispatch."""
self.setup_mock(m, "run_workflow")
manual = self._make_manual_workflow()
non_manual = Workflow.objects.create(name="Non-manual", order=0)
non_manual.triggers.add(
WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
),
)
disabled = Workflow.objects.create(
name="Disabled manual",
order=0,
enabled=False,
)
disabled.triggers.add(
WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
),
)
for label, parameters in (
("missing workflow", {}),
("nonexistent workflow", {"workflow": 9999}),
("no manual trigger", {"workflow": non_manual.id}),
("disabled workflow", {"workflow": disabled.id}),
("non-bool filter", {"workflow": manual.id, "filter_documents": "nope"}),
):
with self.subTest(label):
response = self._run_workflow(parameters)
self.assertEqual(
response.status_code,
status.HTTP_400_BAD_REQUEST,
)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.run_workflow")
def test_run_workflow_requires_ownership(self, m) -> None:
"""A non-owner without document ownership is forbidden (actions mutate docs)."""
self.setup_mock(m, "run_workflow")
user = UserFactory(username="no_perms")
grant_all_global(user)
doc = DocumentFactory(owner=UserFactory(username="other"))
workflow = self._make_manual_workflow()
self.client.force_authenticate(user=user)
response = self._run_workflow({"workflow": workflow.id}, documents=[doc.id])
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.views.bulk_edit.run_workflow")
def test_run_workflow_requires_workflow_permission(self, m) -> None:
"""Owning the documents is not enough without view_workflow permission."""
self.setup_mock(m, "run_workflow")
user = UserFactory(username="no_wf_perms")
grant_global(user, "change_document")
doc = DocumentFactory(owner=user)
workflow = self._make_manual_workflow()
self.client.force_authenticate(user=user)
response = self._run_workflow({"workflow": workflow.id}, documents=[doc.id])
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
m.assert_not_called()
@mock.patch("documents.serialisers.bulk_edit.set_storage_path")
def test_api_set_storage_path(self, m) -> None:
"""
@@ -648,9 +718,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertListEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs["storage_path"], self.sp1.id)
@@ -681,9 +748,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertListEqual(args[0], [self.doc1.id])
self.assertEqual(kwargs["storage_path"], None)
@@ -806,9 +870,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertEqual(args[0], [self.doc2.id])
self.assertEqual(kwargs["storage_path"], self.sp1.id)
@@ -832,9 +893,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc1.id, self.doc3.id, self.doc5.id])
self.assertEqual(kwargs["storage_path"], self.sp1.id)
@@ -871,9 +929,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [first_duplicate.id, second_duplicate.id])
self.assertEqual(kwargs["storage_path"], self.sp1.id)
@@ -910,11 +965,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
get_backend.return_value.search_ids.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value
for key, value in kwargs.items()
if key != "update_callback"
}
self.assertEqual(args[0], [self.doc2.id])
self.assertEqual(kwargs["storage_path"], self.sp1.id)
@@ -1200,8 +1250,8 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertCountEqual(
response.data["selected_document_types"],
[
{"id": self.dt1.id, "document_count": 1},
{"id": self.dt2.id, "document_count": 0},
{"id": self.c1.id, "document_count": 1},
{"id": self.c2.id, "document_count": 0},
],
)
@@ -1291,9 +1341,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
self.assertEqual(len(kwargs["set_permissions"]["view"]["users"]), 2)
@@ -1706,9 +1753,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
self.assertEqual(kwargs["degrees"], 90)
self.assertEqual(kwargs["source_mode"], "latest_version")
@@ -1995,9 +2039,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc2.id, self.doc3.id])
self.assertEqual(kwargs["metadata_document_id"], self.doc3.id)
self.assertEqual(kwargs["source_mode"], "latest_version")
@@ -2205,9 +2246,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc2.id])
self.assertEqual(kwargs["operations"], [{"page": 1}])
self.assertEqual(kwargs["source_mode"], "explicit_selection")
@@ -2553,9 +2591,6 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
self.assertEqual(response.status_code, status.HTTP_200_OK)
m.assert_called_once()
args, kwargs = m.call_args
kwargs = {
key: value for key, value in kwargs.items() if key != "update_callback"
}
self.assertCountEqual(args[0], [self.doc2.id])
self.assertEqual(kwargs["password"], "secret")
self.assertTrue(kwargs["update_document"])
-145
View File
@@ -49,7 +49,6 @@ from documents.models import WorkflowTrigger
from documents.search import get_backend
from documents.signals.handlers import run_workflows
from documents.tests.utils import ConsumeTaskMixin
from documents.views import DocumentViewSet
from paperless_testing.dirs import DirectoriesMixin
from paperless_testing.factories import DocumentFactory
from paperless_testing.factories import TagFactory
@@ -1756,113 +1755,12 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
}
self.assertEqual(counts_by_field[custom_field.id], 1)
def test_selection_data_counts_with_shared_by_filter(self) -> None:
"""
GIVEN:
- Several documents owned by one user with the same correspondent
and document type, only one of which is shared
WHEN:
- Requesting the document list filtered by shared_by__id with
include_selection_data=true
THEN:
- Counts reflect only the shared document, not every document the
owner has with that correspondent or document type, regardless of
the batch size
"""
owner = User.objects.create_user(username="owner")
other = User.objects.create_user(username="other")
correspondent = Correspondent.objects.create(name="c1")
doc_type = DocumentType.objects.create(name="dt1")
tag = Tag.objects.create(name="tag")
docs = [
Document.objects.create(
checksum=str(i),
owner=owner,
correspondent=correspondent,
document_type=doc_type,
)
for i in range(3)
]
for doc in docs:
doc.tags.add(tag)
grant_object(other, docs[0], "view_document")
for batch_size in (10_000, 1):
with (
self.subTest(batch_size=batch_size),
mock.patch.object(
DocumentViewSet,
"selection_data_batch_size",
batch_size,
),
):
response = self.client.get(
f"/api/documents/?shared_by__id={owner.id}"
"&include_selection_data=true",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 1)
selection_data = response.data["selection_data"]
for key, obj in (
("selected_correspondents", correspondent),
("selected_document_types", doc_type),
("selected_tags", tag),
):
counts = {
item["id"]: item["document_count"]
for item in selection_data[key]
}
self.assertEqual(counts[obj.id], 1, key)
def test_selection_data_counts_documents_once_when_filter_joins(self) -> None:
"""
GIVEN:
- A document matching a multi-valued filter through two tags
WHEN:
- Requesting the document list with include_selection_data=true
THEN:
- The document is counted once per correspondent, not once per
matching tag, regardless of the batch size
"""
correspondent = Correspondent.objects.create(name="c1")
tag_a = Tag.objects.create(name="a")
tag_b = Tag.objects.create(name="b")
both = Document.objects.create(checksum="1", correspondent=correspondent)
both.tags.add(tag_a, tag_b)
only_a = Document.objects.create(checksum="2", correspondent=correspondent)
only_a.tags.add(tag_a)
for batch_size in (10_000, 1):
with (
self.subTest(batch_size=batch_size),
mock.patch.object(
DocumentViewSet,
"selection_data_batch_size",
batch_size,
),
):
response = self.client.get(
f"/api/documents/?tags__id__in={tag_a.id},{tag_b.id}"
"&include_selection_data=true",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
counts = {
item["id"]: item["document_count"]
for item in response.data["selection_data"][
"selected_correspondents"
]
}
self.assertEqual(counts[correspondent.id], 2)
def test_statistics(self) -> None:
doc1 = Document.objects.create(
title="none1",
checksum="A",
mime_type="application/pdf",
content="abc",
archive_serial_number=42,
)
Document.objects.create(
title="none2",
@@ -1906,7 +1804,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["correspondent_count"], 2)
self.assertEqual(response.data["document_type_count"], 1)
self.assertEqual(response.data["storage_path_count"], 2)
self.assertEqual(response.data["current_asn"], 42)
def test_statistics_excludes_document_versions(self) -> None:
root = Document.objects.create(
@@ -1914,7 +1811,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
checksum="A",
mime_type="application/pdf",
content="root",
archive_serial_number=42,
)
version = Document.objects.create(
title="version",
@@ -1923,7 +1819,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
content="version",
root_document=root,
version_index=1,
archive_serial_number=99,
)
tag_inbox = Tag.objects.create(name="t1", is_inbox_tag=True)
version.tags.add(tag_inbox)
@@ -1934,7 +1829,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["documents_total"], 1)
self.assertEqual(response.data["documents_inbox"], 0)
self.assertEqual(response.data["character_count"], 4)
self.assertEqual(response.data["current_asn"], 42)
self.assertEqual(
response.data["document_file_type_counts"][0]["mime_type_count"],
1,
@@ -1948,45 +1842,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
self.assertEqual(response.data["documents_inbox"], None)
self.assertEqual(response.data["inbox_tags"], None)
def test_statistics_current_asn_respects_document_permissions(self) -> None:
viewer = UserFactory()
grant_global(viewer, "view_document")
DocumentFactory(owner=viewer, archive_serial_number=12)
shared = DocumentFactory(owner=self.user, archive_serial_number=23)
grant_object(viewer, shared, "view_document")
DocumentFactory(owner=self.user, archive_serial_number=99)
self.client.force_authenticate(user=viewer)
response = self.client.get("/api/statistics/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["current_asn"], 23)
def test_statistics_current_asn_without_visible_documents(self) -> None:
DocumentFactory(owner=self.user, archive_serial_number=99)
for permissions in ((), ("view_document",)):
viewer = UserFactory()
grant_global(viewer, *permissions)
self.client.force_authenticate(user=viewer)
with self.subTest(permissions=permissions):
response = self.client.get("/api/statistics/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["current_asn"], 0)
def test_statistics_current_asn_with_global_statistics_permission(self) -> None:
viewer = UserFactory()
grant_global(viewer, "view_global_statistics")
DocumentFactory(owner=self.user, archive_serial_number=99)
self.client.force_authenticate(user=viewer)
response = self.client.get("/api/statistics/")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["current_asn"], 99)
def test_statistics_multiple_users(self) -> None:
"""
GIVEN:
+29 -31
View File
@@ -25,7 +25,6 @@ from documents.models import Tag
from documents.models import Workflow
from documents.search import get_backend
from documents.search import reset_backend
from documents.views import DocumentViewSet
from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
from paperless_testing.dirs import DirectoriesMixin
@@ -346,39 +345,38 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
storage_path = StoragePath.objects.create(name="sp1")
tag = Tag.objects.create(name="tag")
for checksum, title in (
("A", "bank statement"),
("B", "bank account"),
("C", "credit card"),
):
doc = Document.objects.create(
title=title,
checksum=checksum,
correspondent=correspondent,
document_type=doc_type,
storage_path=storage_path,
)
doc.tags.add(tag)
get_backend().add_or_update(doc)
matching_doc = Document.objects.create(
title="bank statement",
content="bank content",
checksum="A",
correspondent=correspondent,
document_type=doc_type,
storage_path=storage_path,
)
matching_doc.tags.add(tag)
with mock.patch.object(DocumentViewSet, "selection_data_batch_size", 1):
response = self.client.get(
"/api/documents/?query=bank&page_size=1&include_selection_data=true",
)
get_backend().add_or_update(matching_doc)
response = self.client.get(
"/api/documents/?query=bank&include_selection_data=true",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 2)
self.assertEqual(len(response.data["results"]), 1)
for key, pk in (
("selected_correspondents", correspondent.pk),
("selected_document_types", doc_type.pk),
("selected_storage_paths", storage_path.pk),
("selected_tags", tag.pk),
):
self.assertEqual(
response.data["selection_data"][key],
[{"id": pk, "document_count": 2}],
)
self.assertIn("selection_data", response.data)
selected_correspondent = next(
item
for item in response.data["selection_data"]["selected_correspondents"]
if item["id"] == correspondent.id
)
selected_tag = next(
item
for item in response.data["selection_data"]["selected_tags"]
if item["id"] == tag.id
)
self.assertEqual(selected_correspondent["document_count"], 1)
self.assertEqual(selected_tag["document_count"], 1)
def test_search_custom_field_ordering(self) -> None:
custom_field = CustomField.objects.create(
@@ -149,10 +149,10 @@ class TestPostSelectionPathsEnforceTheCap:
assert str(_MAX_QUERY_LENGTH + 1) in message
parse_spy.assert_not_called()
@mock.patch("documents.views.execute_bulk_operation.apply_async")
@mock.patch("documents.bulk_edit.bulk_update_documents.apply_async")
def test_bulk_edit_query_at_exactly_the_cap_is_accepted(
self,
enqueue_mock: mock.MagicMock,
bulk_update_task_mock: mock.MagicMock,
admin_client: APIClient,
searchable_document: Document,
) -> None:
@@ -163,12 +163,12 @@ class TestPostSelectionPathsEnforceTheCap:
- Its `filters.query` is exactly `_MAX_QUERY_LENGTH` characters
long
THEN:
- The cap check accepts it and queues the metadata operation.
Mock the executor dispatch so this exercises query validation
without requiring a broker.
- The cap check accepts it and the request reaches the real
bulk-edit method (its Celery dispatch is mocked out here,
same as every other bulk-edit test, since nothing here is
testing that method itself)
"""
query = "a" * _MAX_QUERY_LENGTH
enqueue_mock.return_value.id = "query-cap-task"
response = admin_client.post(
"/api/documents/bulk_edit/",
@@ -183,11 +183,6 @@ class TestPostSelectionPathsEnforceTheCap:
)
assert response.status_code == status.HTTP_200_OK
assert response.data == {"result": "OK", "task_id": "query-cap-task"}
enqueue_mock.assert_called_once()
assert (
enqueue_mock.call_args.kwargs["kwargs"]["operation"] == "set_document_type"
)
def test_bulk_download_query_one_over_the_cap_is_a_400(
self,
+41
View File
@@ -100,6 +100,47 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
self.action.assign_correspondent.pk,
)
def test_api_get_workflows_filter_trigger_type(self) -> None:
"""
GIVEN:
- A consumption workflow and a manual workflow
WHEN:
- Workflows are requested filtered by trigger_type
THEN:
- Only workflows with a matching trigger are returned
"""
manual_trigger = WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
)
manual_workflow = Workflow.objects.create(name="Manual", order=1)
manual_workflow.triggers.add(manual_trigger)
response = self.client.get(
f"{self.ENDPOINT}?trigger_type={WorkflowTrigger.WorkflowTriggerType.MANUAL}",
format="json",
)
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 1)
self.assertEqual(response.data["results"][0]["id"], manual_workflow.id)
def test_api_get_workflows_filter_enabled(self) -> None:
"""
GIVEN:
- An enabled workflow and a disabled workflow
WHEN:
- Workflows are requested filtered by enabled
THEN:
- Only enabled workflows are returned
"""
Workflow.objects.create(name="Disabled", order=1, enabled=False)
response = self.client.get(f"{self.ENDPOINT}?enabled=true", format="json")
self.assertEqual(response.status_code, status.HTTP_200_OK)
self.assertEqual(response.data["count"], 1)
self.assertEqual(response.data["results"][0]["id"], self.workflow.id)
def test_api_get_workflow_actions_ordered(self) -> None:
"""
GIVEN:
-307
View File
@@ -1,307 +0,0 @@
"""Request acceptance, worker execution, and task tracking."""
import uuid
from unittest import mock
import pytest
from auditlog.models import LogEntry
from django.contrib.auth.models import User
from rest_framework.exceptions import PermissionDenied
from rest_framework.test import APIClient
from documents.bulk_operations import execute_bulk_operation
from documents.models import Correspondent
from documents.models import Document
from documents.models import PaperlessTask
from documents.signals.handlers import before_task_publish_handler
from documents.signals.handlers import task_failure_handler
from documents.signals.handlers import task_postrun_handler
from documents.signals.handlers import task_revoked_handler
from paperless_testing.factories import UserFactory
@pytest.fixture
def bulk_setup(paperless_dirs):
user = UserFactory(superuser=True)
client = APIClient()
client.force_authenticate(user)
doc = Document.objects.create(title="Example", checksum="bulk-op-example")
correspondent = Correspondent.objects.create(name="Example correspondent")
return user, client, doc, correspondent
@pytest.mark.django_db
@pytest.mark.parametrize(
"endpoint,operation,parameters",
[
("bulk_edit", "set_correspondent", {"correspondent": None}),
],
)
def test_request_only_queues(bulk_setup, endpoint, operation, parameters):
user, client, doc, _ = bulk_setup
task_id = str(uuid.uuid4())
data = {"documents": [doc.pk], "from_webui": True}
if endpoint == "bulk_edit":
data.update(method=operation, parameters=parameters)
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
enqueue.return_value.id = task_id
response = client.post(f"/api/documents/{endpoint}/", data, format="json")
assert response.status_code == 200
assert response.data == {"result": "OK", "task_id": task_id}
assert Document.objects.filter(pk=doc.pk).exists()
assert enqueue.call_args.kwargs["kwargs"] == {
"operation": operation,
"document_ids": [doc.pk],
"parameters": parameters,
"user_id": user.pk,
}
assert enqueue.call_args.kwargs["headers"]["trigger_source"] == "web_ui"
@pytest.mark.django_db
def test_metadata_task_includes_audit_and_followup(bulk_setup, settings):
user, _, doc, correspondent = bulk_setup
settings.AUDIT_LOG_ENABLED = True
with (
mock.patch("documents.bulk_operations.bulk_update_documents") as followup,
mock.patch(
"documents.bulk_edit.bulk_update_documents.apply_async",
) as extra_task,
):
def check_followup(ids):
doc.refresh_from_db()
assert doc.correspondent_id == correspondent.pk
assert LogEntry.objects.filter(object_pk=str(doc.pk), actor=user).exists()
assert ids == [doc.pk]
followup.side_effect = check_followup
result = execute_bulk_operation(
"set_correspondent",
[doc.pk],
{"correspondent": correspondent.pk},
user.pk,
)
followup.assert_called_once()
extra_task.assert_not_called()
assert result["document_count"] == 1
@pytest.mark.django_db
def test_permissions_rechecked_on_worker(bulk_setup):
user, _, doc, correspondent = bulk_setup
User.objects.filter(pk=user.pk).update(is_superuser=False)
with pytest.raises(PermissionDenied):
execute_bulk_operation(
"set_correspondent",
[doc.pk],
{"correspondent": correspondent.pk},
user.pk,
)
doc.refresh_from_db()
assert doc.correspondent_id is None
@pytest.mark.django_db
def test_queue_failure_does_not_change_documents(bulk_setup):
_, client, doc, correspondent = bulk_setup
with mock.patch(
"documents.views.execute_bulk_operation.apply_async",
side_effect=RuntimeError("broker unavailable"),
):
response = client.post(
"/api/documents/bulk_edit/",
{
"documents": [doc.pk],
"method": "set_correspondent",
"parameters": {"correspondent": correspondent.pk},
},
format="json",
)
assert response.status_code == 400
doc.refresh_from_db()
assert doc.correspondent_id is None
@pytest.mark.django_db
@pytest.mark.parametrize("failed", [False, True])
def test_tracked_outcome_records_initiator_and_result(bulk_setup, failed):
user, _, doc, _ = bulk_setup
task_id = str(uuid.uuid4())
before_task_publish_handler(
headers={
"task": "documents.bulk_operations.execute_bulk_operation",
"id": task_id,
},
body=(
(),
{
"operation": "set_permissions",
"document_ids": [doc.pk],
"user_id": user.pk,
},
{},
),
)
task = PaperlessTask.objects.get(task_id=task_id)
assert task.owner_id == user.pk
assert task.task_type == PaperlessTask.TaskType.BULK_UPDATE
if failed:
task_failure_handler(task_id=task_id, exception=ValueError("Could not update"))
else:
task_postrun_handler(
task_id=task_id,
state="SUCCESS",
retval={"document_count": 1},
)
task.refresh_from_db()
assert task.status == ("failure" if failed else "success")
if failed:
assert task.result_data["error_message"] == "Could not update"
else:
assert task.result_data == {"document_count": 1}
@pytest.mark.django_db
def test_large_filtered_selection_is_one_task(bulk_setup):
_, client, _, _ = bulk_setup
Document.objects.bulk_create(
[
Document(title="bulk-large-selection", checksum=f"large-{i}")
for i in range(10_000)
],
)
with mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue:
enqueue.return_value.id = str(uuid.uuid4())
response = client.post(
"/api/documents/bulk_edit/",
{
"all": True,
"filters": {"title__icontains": "bulk-large-selection"},
"method": "set_correspondent",
"parameters": {"correspondent": None},
},
format="json",
)
assert response.status_code == 200
enqueue.assert_called_once()
assert len(enqueue.call_args.kwargs["kwargs"]["document_ids"]) == 10_000
@pytest.mark.django_db
@pytest.mark.parametrize(
("outcome", "expected_status", "expected_error"),
[
("success", "success", ""),
("failure", "failure", "Could not update"),
(
"expired",
"revoked",
"Task expired before execution. Check worker logs for details.",
),
],
)
def test_tracked_outcome_notifies_only_initiator(
bulk_setup,
outcome,
expected_status,
expected_error,
):
user, _, doc, _ = bulk_setup
task_id = str(uuid.uuid4())
before_task_publish_handler(
headers={
"task": execute_bulk_operation.name,
"id": task_id,
},
body=(
(),
{
"operation": "set_permissions",
"document_ids": [doc.pk],
"user_id": user.pk,
},
{},
),
)
task = PaperlessTask.objects.get(task_id=task_id)
assert task.owner_id == user.pk
assert task.task_type == PaperlessTask.TaskType.BULK_UPDATE
with mock.patch("documents.signals.handlers.async_to_sync") as send:
if outcome == "expired":
task_revoked_handler(
sender=execute_bulk_operation,
request=mock.Mock(id=task_id),
expired=True,
)
elif outcome == "failure":
task_failure_handler(
sender=execute_bulk_operation,
task_id=task_id,
exception=ValueError("Could not update"),
)
else:
task_postrun_handler(
task=execute_bulk_operation,
task_id=task_id,
state="SUCCESS",
retval={"document_count": 1},
)
group, event = send.return_value.call_args.args
assert group == f"bulk_operations_{user.pk}"
assert event["data"]["status"] == expected_status
assert event["data"]["error"] == expected_error
assert event["data"]["task_id"] == task_id
@pytest.mark.anyio
@pytest.mark.parametrize("anyio_backend", ["asyncio"])
async def test_completion_socket_is_private(anyio_backend):
from types import SimpleNamespace
from channels.layers import get_channel_layer
from channels.testing import WebsocketCommunicator
from paperless.consumers import StatusConsumer
own = WebsocketCommunicator(StatusConsumer.as_asgi(), "/ws/status/")
other = WebsocketCommunicator(StatusConsumer.as_asgi(), "/ws/status/")
own.scope["user"] = SimpleNamespace(id=1, is_authenticated=True)
other.scope["user"] = SimpleNamespace(id=2, is_authenticated=True)
assert (await own.connect())[0]
assert (await other.connect())[0]
try:
event = {
"type": "bulk_operation_status",
"data": {"task_id": "example", "status": "success"},
}
await get_channel_layer().group_send("bulk_operations_1", event)
assert await own.receive_json_from() == event
assert await other.receive_nothing(timeout=0.02)
finally:
await own.disconnect()
await other.disconnect()
@pytest.mark.django_db
@pytest.mark.parametrize("endpoint", ["delete", "bulk_edit"])
def test_trash_remains_synchronous(bulk_setup, endpoint):
_, client, doc, _ = bulk_setup
data = {"documents": [doc.pk]}
if endpoint == "bulk_edit":
data.update(method="delete", parameters={})
with (
mock.patch("documents.views.execute_bulk_operation.apply_async") as enqueue,
mock.patch("documents.search.get_backend"),
):
response = client.post(
f"/api/documents/{endpoint}/",
data,
format="json",
HTTP_ACCEPT="application/json; version=9",
)
assert response.status_code == 200
assert response.data == {"result": "OK"}
assert not Document.objects.filter(pk=doc.pk).exists()
assert Document.deleted_objects.filter(pk=doc.pk).exists()
enqueue.assert_not_called()
-36
View File
@@ -5,11 +5,8 @@ from unittest import mock
import pytest
from django.conf import settings
from django.contrib.sessions.models import Session
from django.db import connection
from django.test import TestCase
from django.test import override_settings
from django.test.utils import CaptureQueriesContext
from django.utils import timezone
from documents import tasks
@@ -615,36 +612,3 @@ class TestApplyAISuggestionsTask(DirectoriesMixin, TestCase):
apply_suggestions.assert_not_called()
self.assertIn("no longer exists", "".join(cm.output))
@pytest.mark.django_db
class TestClearExpiredSessions:
def test_clears_only_expired_sessions_in_batches(self) -> None:
"""
GIVEN:
- More than one batch of expired sessions and one still-valid session
WHEN:
- The clear_expired_sessions task runs
THEN:
- Only expired sessions are removed, using multiple deletes
"""
now = timezone.now()
Session.objects.bulk_create(
Session(
session_key=f"expired-{i}",
session_data="",
expire_date=now - timedelta(days=1),
)
for i in range(501)
)
Session.objects.create(
session_key="valid",
session_data="",
expire_date=now + timedelta(days=1),
)
with CaptureQueriesContext(connection) as queries:
tasks.clear_expired_sessions()
assert set(Session.objects.values_list("session_key", flat=True)) == {"valid"}
assert sum(query["sql"].startswith("DELETE FROM") for query in queries) == 2
+77
View File
@@ -2288,6 +2288,83 @@ class TestWorkflows(
doc.refresh_from_db()
self.assertEqual(doc.owner, self.user2)
def _make_manual_workflow(self, *, require_tag: Tag | None = None) -> Workflow:
trigger = WorkflowTrigger.objects.create(
type=WorkflowTrigger.WorkflowTriggerType.MANUAL,
)
if require_tag is not None:
trigger.filter_has_all_tags.add(require_tag)
action = WorkflowAction.objects.create(assign_owner=self.user2)
workflow = Workflow.objects.create(name="Manual workflow", order=0)
workflow.triggers.add(trigger)
workflow.actions.add(action)
return workflow
@mock.patch("documents.tasks.send_websocket_document_updated")
def test_manual_workflow_run_applies_and_notifies(self, mock_ws) -> None:
"""A manual run applies the workflow's actions and pushes a websocket update."""
workflow = self._make_manual_workflow()
doc = DocumentFactory()
tasks.run_workflows_for_documents([doc.pk], workflow.pk)
doc.refresh_from_db()
self.assertEqual(doc.owner, self.user2)
self.assertEqual(mock_ws.call_args.kwargs["document"].pk, doc.pk)
def test_manual_workflow_run_respects_filters(self) -> None:
"""With filter_documents=True, a document failing the trigger filters is skipped."""
workflow = self._make_manual_workflow(require_tag=self.t1)
doc = DocumentFactory()
tasks.run_workflows_for_documents([doc.pk], workflow.pk, filter_documents=True)
doc.refresh_from_db()
self.assertIsNone(doc.owner)
def test_manual_workflow_run_force_ignores_filters(self) -> None:
"""With filter_documents=False, actions run regardless of the trigger filters."""
workflow = self._make_manual_workflow(require_tag=self.t1)
doc = DocumentFactory()
tasks.run_workflows_for_documents([doc.pk], workflow.pk, filter_documents=False)
doc.refresh_from_db()
self.assertEqual(doc.owner, self.user2)
def test_manual_workflow_run_missing_workflow(self) -> None:
"""A missing workflow id logs a warning rather than raising."""
doc = DocumentFactory()
with self.assertLogs("paperless.tasks", level="WARNING"):
tasks.run_workflows_for_documents([doc.pk], 9999)
@mock.patch("documents.tasks.send_websocket_document_updated")
@mock.patch("documents.tasks.run_workflows")
def test_manual_workflow_run_continues_after_error(
self,
mock_run_workflows,
mock_ws,
) -> None:
"""A failure on one document does not abort the run for the others."""
workflow = self._make_manual_workflow()
doc1 = DocumentFactory()
doc2 = DocumentFactory()
def side_effect(*args, document, **kwargs):
if document.pk == doc1.pk:
raise Exception("boom")
mock_run_workflows.side_effect = side_effect
with self.assertLogs("paperless.tasks", level="ERROR"):
tasks.run_workflows_for_documents([doc1.pk, doc2.pk], workflow.pk)
# Both documents were attempted; the websocket update only fired for the
# one that did not raise.
self.assertEqual(mock_run_workflows.call_count, 2)
ws_doc_ids = [c.kwargs["document"].pk for c in mock_ws.call_args_list]
self.assertEqual(ws_doc_ids, [doc2.pk])
@mock.patch("documents.tasks.send_websocket_document_updated")
def test_workflow_scheduled_trigger_sends_websocket_update(
self,
+219 -114
View File
@@ -5,14 +5,11 @@ import platform
import re
import tempfile
import zipfile
from collections import Counter
from collections import deque
from collections.abc import Callable
from datetime import datetime
from datetime import timedelta
from http import HTTPStatus
from inspect import signature
from itertools import islice
from pathlib import Path
from time import mktime
from time import sleep
@@ -107,9 +104,6 @@ from documents import bulk_edit
from documents.bulk_download import ArchiveOnlyStrategy
from documents.bulk_download import OriginalAndArchiveStrategy
from documents.bulk_download import OriginalsOnlyStrategy
from documents.bulk_operations import QUEUED_OPERATIONS
from documents.bulk_operations import execute_bulk_operation
from documents.bulk_operations import snapshot_field
from documents.caching import get_llm_suggestion_cache
from documents.caching import get_metadata_cache
from documents.caching import get_suggestion_cache
@@ -146,6 +140,7 @@ from documents.filters import ShareLinkFilterSet
from documents.filters import StoragePathFilterSet
from documents.filters import TagFilterSet
from documents.filters import TitleContentFilter
from documents.filters import WorkflowFilterSet
from documents.mail import EmailAttachment
from documents.mail import send_email
from documents.matching import match_correspondents
@@ -174,10 +169,10 @@ from documents.permissions import PaperlessNotePermissions
from documents.permissions import PaperlessObjectPermissions
from documents.permissions import TrashPermissions
from documents.permissions import ViewDocumentsPermissions
from documents.permissions import annotate_document_count_by_ids
from documents.permissions import annotate_document_count_for_related_queryset
from documents.permissions import get_document_count_filter_for_user
from documents.permissions import get_permission_grantees
from documents.permissions import has_document_operation_permissions
from documents.permissions import has_global_statistics_permission
from documents.permissions import has_perms_owner_aware
from documents.permissions import has_system_status_permission
@@ -1020,80 +1015,71 @@ class DocumentViewSet(
"custom_field_",
)
# Bound the IN clauses used for search results and through-table lookups.
selection_data_batch_size = 10_000
def _get_selection_data_for_queryset(self, queryset):
# Resolve once instead of once per model below. `queryset` can carry an
# arbitrarily expensive WHERE clause (user filters plus the permission
# filter); re-embedding it as a subquery inside 5 separate Count(...)
# calls forces the database to re-evaluate that whole thing 5 times, and
# -- for FK relations especially -- can defeat semi-join planning
# entirely at scale. A concrete id list is cheap to reuse.
# order_by() drops the default/user ordering -- irrelevant for a plain
# id list, but left in place it forces a sort over the full filtered
# set before the ids can even be collected.
document_ids = list(queryset.order_by().values_list("pk", flat=True))
def _get_selection_data_for_queryset(
self,
queryset: QuerySet[Document],
*,
search_ids: list[int] | None = None,
) -> dict[str, list[dict[str, int]]]:
# Keep permission and filter predicates at document granularity.
# Reusing the filtered queryset as a subquery for each count is slow.
rows = (
queryset.order_by()
.prefetch_related(None)
.values_list(
"pk",
"correspondent_id",
"document_type_id",
"storage_path_id",
)
correspondents = Correspondent.objects.annotate(
document_count=Count(
"documents",
filter=Q(documents__id__in=document_ids),
distinct=True,
),
)
document_types = DocumentType.objects.annotate(
document_count=Count(
"documents",
filter=Q(documents__id__in=document_ids),
distinct=True,
),
)
storage_paths = StoragePath.objects.annotate(
document_count=Count(
"documents",
filter=Q(documents__id__in=document_ids),
distinct=True,
),
)
# Tag and CustomField reach Document through an M2M/through-model table;
# a plain Count(filter=...) there is a much more expensive plan than the
# FK relations above once the bridge table is large -- see
# annotate_document_count_by_ids() for why.
tags = annotate_document_count_by_ids(
Tag.objects.all(),
through_model=Document.tags.through,
related_object_field="tag_id",
document_ids=document_ids,
)
custom_fields = annotate_document_count_by_ids(
CustomField.objects.all(),
through_model=CustomFieldInstance,
related_object_field="field_id",
document_ids=document_ids,
)
batch_size = self.selection_data_batch_size
def document_rows():
if search_ids is None:
yield from rows.iterator()
else:
for start in range(0, len(search_ids), batch_size):
yield from rows.filter(
pk__in=search_ids[start : start + batch_size],
).iterator()
correspondents: Counter[int | None] = Counter()
document_types: Counter[int | None] = Counter()
storage_paths: Counter[int | None] = Counter()
document_ids: set[int] = set()
for pk, correspondent, document_type, storage_path in document_rows():
# Filters that join multi-valued relations can repeat a document.
if pk in document_ids:
continue
document_ids.add(pk)
correspondents[correspondent] += 1
document_types[document_type] += 1
storage_paths[storage_path] += 1
tags: Counter[int | None] = Counter()
custom_fields: Counter[int | None] = Counter()
ids = iter(document_ids)
while batch := list(islice(ids, batch_size)):
for model, field, counts in (
(Document.tags.through, "tag_id", tags),
(CustomFieldInstance, "field_id", custom_fields),
):
counts.update(
dict(
model.objects.filter(document_id__in=batch)
.order_by()
.values_list(field)
.annotate(document_count=Count("document_id")),
),
)
def selection(model: type[Model], counts: Counter[int | None]):
return [
{"id": pk, "document_count": counts[pk]}
for pk in model._default_manager.values_list("pk", flat=True)
]
return {
"selected_correspondents": selection(Correspondent, correspondents),
"selected_tags": selection(Tag, tags),
"selected_document_types": selection(DocumentType, document_types),
"selected_storage_paths": selection(StoragePath, storage_paths),
"selected_custom_fields": selection(CustomField, custom_fields),
"selected_correspondents": [
{"id": t.id, "document_count": t.document_count} for t in correspondents
],
"selected_tags": [
{"id": t.id, "document_count": t.document_count} for t in tags
],
"selected_document_types": [
{"id": t.id, "document_count": t.document_count} for t in document_types
],
"selected_storage_paths": [
{"id": t.id, "document_count": t.document_count} for t in storage_paths
],
"selected_custom_fields": [
{"id": t.id, "document_count": t.document_count} for t in custom_fields
],
}
@classmethod
@@ -2740,10 +2726,13 @@ class UnifiedSearchViewSet(DocumentViewSet):
if get_boolean(
str(request.query_params.get("include_selection_data", "false")),
):
# NOTE: pk__in=ordered_ids generates a large SQL IN clause
# for big result sets. Acceptable today but may need a temp
# table or chunked approach if selection_data becomes slow
# at scale (tens of thousands of matching documents).
response.data["selection_data"] = (
self._get_selection_data_for_queryset(
filtered_qs,
search_ids=result.ordered_ids,
filtered_qs.filter(pk__in=result.ordered_ids),
)
)
return response
@@ -2987,26 +2976,99 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
"merge_as_versions",
}
def _queue_bulk_operation(self, method, documents, parameters, from_webui):
if "update_callback" in parameters:
raise ValueError("Unsupported bulk operation parameter: update_callback")
signature(method).bind(documents, **parameters)
task = execute_bulk_operation.apply_async(
kwargs={
"operation": method.__name__,
"document_ids": documents,
"parameters": parameters,
"user_id": self.request.user.pk,
},
headers={
"trigger_source": (
PaperlessTask.TriggerSource.WEB_UI
if from_webui
else PaperlessTask.TriggerSource.API_UPLOAD
),
},
def _has_document_permissions(
self,
*,
user: User,
documents: list[int],
method,
parameters: dict[str, Any],
) -> bool:
if user.is_superuser:
return True
root_docs = {
get_root_document(doc)
for doc in Document.objects.select_related(
"owner",
"root_document__owner",
).filter(pk__in=documents)
}
user_is_owner_of_all_documents = all(
(doc.owner == user or doc.owner is None) for doc in root_docs
)
return Response({"result": "OK", "task_id": task.id})
# check global and object permissions for all documents
has_perms = (
user.has_perm(
"documents.change_document",
)
and not Document.global_objects.filter(pk__in=documents)
.exclude(pk__in=permitted_document_ids(user, perm="change_document"))
.exists()
)
# check ownership for methods that change original document
if (
(
has_perms
and method
in [
bulk_edit.set_permissions,
bulk_edit.delete,
bulk_edit.rotate,
bulk_edit.delete_pages,
bulk_edit.edit_pdf,
bulk_edit.merge_as_versions,
bulk_edit.remove_password,
bulk_edit.run_workflow,
]
)
or (
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (method == bulk_edit.edit_pdf and parameters.get("update_document"))
):
has_perms = has_perms and user_is_owner_of_all_documents
# check global add permissions for methods that create documents
if (
has_perms
and (
method in [bulk_edit.split, bulk_edit.merge]
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.add_document")
):
has_perms = False
# check global delete permissions for methods that delete documents
if (
has_perms
and (
method == bulk_edit.delete
# Sources stop being documents of their own, and removing one
# again afterwards needs delete_document
or method == bulk_edit.merge_as_versions
or (
method in [bulk_edit.merge, bulk_edit.split]
and parameters.get("delete_originals")
)
or (
method in [bulk_edit.edit_pdf, bulk_edit.remove_password]
and parameters.get("delete_original")
and not parameters.get("update_document")
)
)
and not user.has_perm("documents.delete_document")
):
has_perms = False
return has_perms
def _execute_document_action(
self,
@@ -3043,7 +3105,7 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
else PaperlessTask.TriggerSource.API_UPLOAD
)
if not has_document_operation_permissions(
if not self._has_document_permissions(
user=user,
documents=documents,
method=method,
@@ -3074,20 +3136,68 @@ class DocumentOperationPermissionMixin(PassUserMixin, DocumentSelectionMixin):
name="BulkEditDocumentsResult",
fields={
"result": serializers.CharField(),
"task_id": serializers.CharField(required=False),
},
),
},
),
)
class BulkEditView(DocumentOperationPermissionMixin):
LEGACY_MODIFIED_FIELDS = {
MODIFIED_FIELD_BY_METHOD = {
"set_correspondent": "correspondent",
"set_document_type": "document_type",
"set_storage_path": "storage_path",
"add_tag": "tags",
"remove_tag": "tags",
"modify_tags": "tags",
"modify_custom_fields": "custom_fields",
"set_permissions": None,
"delete": "deleted_at",
# These operations create new documents/versions no longer altering
# fields on the selected document in place
"rotate": None,
"delete_pages": None,
"split": None,
"merge": None,
"edit_pdf": None,
"reprocess": "checksum",
"remove_password": None,
"run_workflow": None,
}
serializer_class = BulkEditSerializer
@staticmethod
def _snapshot_field(doc_ids: list[int], field: str) -> dict[int, Any]:
"""
Returns each document's current value of field, for the audit log.
Tags and custom fields are one row per value, so they are gathered
into a sorted list of pks per document (empty when there are none).
Reading them through Document.values() instead would join those rows
and return one arbitrary value per document.
"""
if field == "tags":
rows = (
Document.tags.through.objects.filter(document_id__in=doc_ids)
.order_by("tag_id")
.values_list("document_id", "tag_id")
)
elif field == "custom_fields":
rows = (
CustomFieldInstance.objects.filter(document_id__in=doc_ids)
.order_by("pk")
.values_list("document_id", "pk")
)
else:
return dict(
Document.objects.filter(pk__in=doc_ids).values_list("pk", field),
)
values: dict[int, list[int]] = {doc_id: [] for doc_id in doc_ids}
for doc_id, pk in rows:
values[doc_id].append(pk)
return values
def post(self, request, *args, **kwargs):
request_method = request.data.get("method")
api_version = int(request.version or settings.REST_FRAMEWORK["DEFAULT_VERSION"])
@@ -3123,7 +3233,7 @@ class BulkEditView(DocumentOperationPermissionMixin):
if from_webui
else PaperlessTask.TriggerSource.API_UPLOAD
)
if not has_document_operation_permissions(
if not self._has_document_permissions(
user=user,
documents=documents,
method=method,
@@ -3132,21 +3242,14 @@ class BulkEditView(DocumentOperationPermissionMixin):
return HttpResponseForbidden("Insufficient permissions")
try:
if method.__name__ in QUEUED_OPERATIONS:
return self._queue_bulk_operation(
method,
documents,
parameters,
from_webui,
)
modified_field = self.LEGACY_MODIFIED_FIELDS.get(method.__name__)
modified_field = self.MODIFIED_FIELD_BY_METHOD.get(method.__name__, None)
if settings.AUDIT_LOG_ENABLED and modified_field:
old_values = snapshot_field(documents, modified_field)
old_values = self._snapshot_field(documents, modified_field)
result = method(documents, **parameters)
if settings.AUDIT_LOG_ENABLED and modified_field:
new_values = snapshot_field(documents, modified_field)
new_values = self._snapshot_field(documents, modified_field)
for doc in Document.objects.filter(pk__in=documents):
LogEntry.objects.log_create(
instance=doc,
@@ -3971,7 +4074,7 @@ class StatisticsView(GenericAPIView[Any]):
for row in mime_type_stats
]
current_asn = documents.aggregate(
current_asn = Document.objects.aggregate(
Max("archive_serial_number", default=0),
).get(
"archive_serial_number__max",
@@ -5089,6 +5192,8 @@ class WorkflowViewSet(ModelViewSet[Workflow]):
serializer_class = WorkflowSerializer
pagination_class = StandardPagination
filter_backends = (DjangoFilterBackend,)
filterset_class = WorkflowFilterSet
model = Workflow
+50 -50
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr ""
"Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-11-11 23:50+0000\n"
"POT-Creation-Date: 2026-11-06 20:19+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n"
"Language-Team: English\n"
@@ -1661,8 +1661,8 @@ msgid "workflow runs"
msgstr ""
#: documents/serialisers.py:483 documents/serialisers.py:844
#: documents/serialisers.py:2850 documents/views.py:354 documents/views.py:620
#: documents/views.py:2760 paperless_mail/serialisers.py:156
#: documents/serialisers.py:2850 documents/views.py:348 documents/views.py:614
#: documents/views.py:2748 paperless_mail/serialisers.py:156
msgid "Insufficient permissions."
msgstr ""
@@ -1702,7 +1702,7 @@ msgstr ""
msgid "Duplicate document identifiers are not allowed."
msgstr ""
#: documents/serialisers.py:2936 documents/views.py:4698
#: documents/serialisers.py:2936 documents/views.py:4798
#, python-format
msgid "Documents not found: %(ids)s"
msgstr ""
@@ -1953,40 +1953,40 @@ msgstr ""
msgid ", "
msgstr ""
#: documents/views.py:309
#: documents/views.py:303
msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr ""
#: documents/views.py:347 documents/views.py:2757
#: documents/views.py:341 documents/views.py:2745
msgid "Invalid more_like_id"
msgstr ""
#: documents/views.py:1684
#: documents/views.py:1669
msgid "Invalid AI configuration."
msgstr ""
#: documents/views.py:1695
#: documents/views.py:1680
msgid "AI backend request timed out."
msgstr ""
#: documents/views.py:1707
#: documents/views.py:1692
msgid "AI backend rejected the request. Check logs for details."
msgstr ""
#: documents/views.py:4714
#: documents/views.py:4814
#, python-format
msgid "Insufficient permissions to share document %(id)s."
msgstr ""
#: documents/views.py:4761
#: documents/views.py:4861
msgid "Bundle is already being processed."
msgstr ""
#: documents/views.py:4827
#: documents/views.py:4927
msgid "The share link bundle is still being prepared. Please try again later."
msgstr ""
#: documents/views.py:4841
#: documents/views.py:4941
msgid "The share link bundle is unavailable."
msgstr ""
@@ -2278,151 +2278,151 @@ msgstr ""
msgid "paperless application settings"
msgstr ""
#: paperless/settings/__init__.py:581
#: paperless/settings/__init__.py:570
msgid "English (US)"
msgstr ""
#: paperless/settings/__init__.py:582
#: paperless/settings/__init__.py:571
msgid "Arabic"
msgstr ""
#: paperless/settings/__init__.py:583
#: paperless/settings/__init__.py:572
msgid "Afrikaans"
msgstr ""
#: paperless/settings/__init__.py:584
#: paperless/settings/__init__.py:573
msgid "Belarusian"
msgstr ""
#: paperless/settings/__init__.py:585
#: paperless/settings/__init__.py:574
msgid "Bulgarian"
msgstr ""
#: paperless/settings/__init__.py:586
#: paperless/settings/__init__.py:575
msgid "Catalan"
msgstr ""
#: paperless/settings/__init__.py:587
#: paperless/settings/__init__.py:576
msgid "Czech"
msgstr ""
#: paperless/settings/__init__.py:588
#: paperless/settings/__init__.py:577
msgid "Danish"
msgstr ""
#: paperless/settings/__init__.py:589
#: paperless/settings/__init__.py:578
msgid "German"
msgstr ""
#: paperless/settings/__init__.py:590
#: paperless/settings/__init__.py:579
msgid "Greek"
msgstr ""
#: paperless/settings/__init__.py:591
#: paperless/settings/__init__.py:580
msgid "English (GB)"
msgstr ""
#: paperless/settings/__init__.py:592
#: paperless/settings/__init__.py:581
msgid "Spanish"
msgstr ""
#: paperless/settings/__init__.py:593
#: paperless/settings/__init__.py:582
msgid "Persian"
msgstr ""
#: paperless/settings/__init__.py:594
#: paperless/settings/__init__.py:583
msgid "Finnish"
msgstr ""
#: paperless/settings/__init__.py:595
#: paperless/settings/__init__.py:584
msgid "French"
msgstr ""
#: paperless/settings/__init__.py:596
#: paperless/settings/__init__.py:585
msgid "Hungarian"
msgstr ""
#: paperless/settings/__init__.py:597
#: paperless/settings/__init__.py:586
msgid "Indonesian"
msgstr ""
#: paperless/settings/__init__.py:598
#: paperless/settings/__init__.py:587
msgid "Italian"
msgstr ""
#: paperless/settings/__init__.py:599
#: paperless/settings/__init__.py:588
msgid "Japanese"
msgstr ""
#: paperless/settings/__init__.py:600
#: paperless/settings/__init__.py:589
msgid "Korean"
msgstr ""
#: paperless/settings/__init__.py:601
#: paperless/settings/__init__.py:590
msgid "Luxembourgish"
msgstr ""
#: paperless/settings/__init__.py:602
#: paperless/settings/__init__.py:591
msgid "Norwegian"
msgstr ""
#: paperless/settings/__init__.py:603
#: paperless/settings/__init__.py:592
msgid "Dutch"
msgstr ""
#: paperless/settings/__init__.py:604
#: paperless/settings/__init__.py:593
msgid "Polish"
msgstr ""
#: paperless/settings/__init__.py:605
#: paperless/settings/__init__.py:594
msgid "Portuguese (Brazil)"
msgstr ""
#: paperless/settings/__init__.py:606
#: paperless/settings/__init__.py:595
msgid "Portuguese"
msgstr ""
#: paperless/settings/__init__.py:607
#: paperless/settings/__init__.py:596
msgid "Romanian"
msgstr ""
#: paperless/settings/__init__.py:608
#: paperless/settings/__init__.py:597
msgid "Russian"
msgstr ""
#: paperless/settings/__init__.py:609
#: paperless/settings/__init__.py:598
msgid "Slovak"
msgstr ""
#: paperless/settings/__init__.py:610
#: paperless/settings/__init__.py:599
msgid "Slovenian"
msgstr ""
#: paperless/settings/__init__.py:611
#: paperless/settings/__init__.py:600
msgid "Serbian"
msgstr ""
#: paperless/settings/__init__.py:612
#: paperless/settings/__init__.py:601
msgid "Swedish"
msgstr ""
#: paperless/settings/__init__.py:613
#: paperless/settings/__init__.py:602
msgid "Turkish"
msgstr ""
#: paperless/settings/__init__.py:614
#: paperless/settings/__init__.py:603
msgid "Ukrainian"
msgstr ""
#: paperless/settings/__init__.py:615
#: paperless/settings/__init__.py:604
msgid "Vietnamese"
msgstr ""
#: paperless/settings/__init__.py:616
#: paperless/settings/__init__.py:605
msgid "Chinese Simplified"
msgstr ""
#: paperless/settings/__init__.py:617
#: paperless/settings/__init__.py:606
msgid "Chinese Traditional"
msgstr ""
-11
View File
@@ -12,17 +12,6 @@ from paperless.models import ModeChoices
from paperless.models import OutputTypeChoices
from paperless.models import RemoteOCRMode
# OCRmyPDF arguments the parser sets itself, pointing into its own temporary
# directory. The parser reads these files back after OCR, so user supplied
# values can only break consumption (or write to arbitrary paths), never help.
OCR_RESERVED_USER_ARGS: frozenset[str] = frozenset(
{
"input_file_or_options",
"output_file",
"sidecar",
},
)
@dataclasses.dataclass
class BaseConfig:
-15
View File
@@ -45,21 +45,12 @@ class StatusConsumer(AsyncWebsocketConsumer):
await self.close()
return
await self.channel_layer.group_add("status_updates", self.channel_name)
await self.channel_layer.group_add(
f"bulk_operations_{self.scope['user'].id}",
self.channel_name,
)
await self.accept()
self._start_heartbeat()
async def disconnect(self, code: int) -> None:
await self._stop_heartbeat()
await self.channel_layer.group_discard("status_updates", self.channel_name)
if self._authenticated():
await self.channel_layer.group_discard(
f"bulk_operations_{self.scope['user'].id}",
self.channel_name,
)
def _start_heartbeat(self) -> None:
self.heartbeat_task = asyncio.create_task(self._heartbeat_loop())
@@ -93,9 +84,3 @@ class StatusConsumer(AsyncWebsocketConsumer):
await self.close()
elif await self._can_view(event["data"]):
await self.send(json.dumps(event))
async def bulk_operation_status(self, event: dict) -> None:
if not self._authenticated():
await self.close()
else:
await self.send(json.dumps(event))
+1 -22
View File
@@ -17,12 +17,10 @@ from django.conf import settings
from PIL import Image
from documents.parsers import ParseError
from documents.parsers import encode_thumbnail_webp
from documents.parsers import make_thumbnail_from_pdf
from documents.utils import copy_file_with_basic_stats
from documents.utils import maybe_override_pixel_limit
from documents.utils import remove_alpha_channel
from paperless.config import OCR_RESERVED_USER_ARGS
from paperless.config import OcrConfig
from paperless.models import CleanChoices
from paperless.models import ModeChoices
@@ -163,10 +161,6 @@ class RasterisedDocumentParser:
# ------------------------------------------------------------------
def get_thumbnail(self, document_path: Path, mime_type: str) -> Path:
if self.archive_path is None and self.is_image(mime_type):
thumbnail_path = self.tempdir / "convert.webp"
encode_thumbnail_webp(Path(document_path), thumbnail_path)
return thumbnail_path
return make_thumbnail_from_pdf(
self.archive_path or Path(document_path),
self.tempdir,
@@ -357,22 +351,7 @@ class RasterisedDocumentParser:
if self.settings.user_args is not None:
try:
reserved = sorted(
OCR_RESERVED_USER_ARGS & self.settings.user_args.keys(),
)
if reserved:
self.log.warning(
f"Ignoring OCR user arguments managed by paperless: "
f"{', '.join(reserved)}",
)
ocrmypdf_args = {
**ocrmypdf_args,
**{
k: v
for k, v in self.settings.user_args.items()
if k not in OCR_RESERVED_USER_ARGS
},
}
ocrmypdf_args = {**ocrmypdf_args, **self.settings.user_args}
except Exception as e:
self.log.warning(
f"There is an issue with PAPERLESS_OCR_USER_ARGS, so "
-8
View File
@@ -19,7 +19,6 @@ from PIL import Image
from rest_framework import serializers
from rest_framework.authtoken.serializers import AuthTokenSerializer
from paperless.config import OCR_RESERVED_USER_ARGS
from paperless.models import ApplicationConfiguration
from paperless.network import validate_outbound_http_url
from paperless.validators import reject_dangerous_svg
@@ -252,13 +251,6 @@ class ApplicationConfigurationSerializer(
def validate_user_args(self, value):
self._require_json_object("user_args", value)
if value is not None:
reserved = sorted(OCR_RESERVED_USER_ARGS & value.keys())
if reserved:
raise serializers.ValidationError(
f"user_args may not set arguments managed by paperless: "
f"{', '.join(reserved)}.",
)
return value
def validate_barcode_tag_mapping(self, value):
+8 -19
View File
@@ -450,34 +450,23 @@ CORS_EXPOSE_HEADERS = [
"Content-Disposition",
]
def _get_allowed_hosts() -> list[str]:
if "PAPERLESS_ALLOWED_HOSTS" in os.environ or os.getenv("PAPERLESS_URL"):
allowed_hosts = get_list_from_env("PAPERLESS_ALLOWED_HOSTS")
else:
allowed_hosts = ["*"]
if allowed_hosts != ["*"]:
# Always allow localhost for the Docker health check.
allowed_hosts.append("localhost")
return allowed_hosts
ALLOWED_HOSTS = _get_allowed_hosts()
ALLOWED_HOSTS = get_list_from_env("PAPERLESS_ALLOWED_HOSTS", default=["*"])
if ALLOWED_HOSTS != ["*"]:
# always allow localhost. Necessary e.g. for healthcheck in docker.
ALLOWED_HOSTS.append("localhost")
def _parse_paperless_url():
global CSRF_TRUSTED_ORIGINS, CORS_ALLOWED_ORIGINS, ALLOWED_HOSTS
url = os.getenv("PAPERLESS_URL")
if url:
hostname = (urlparse(url).hostname or "").removesuffix(".")
if not hostname or hostname == "*" or hostname.startswith("."):
hostname = urlparse(url).hostname
if not hostname:
raise ImproperlyConfigured(
f"PAPERLESS_URL {url!r} has no valid hostname, expected a URL like https://example.com",
f"PAPERLESS_URL {url!r} has no hostname, expected a URL like https://example.com",
)
CSRF_TRUSTED_ORIGINS.append(url)
CORS_ALLOWED_ORIGINS.append(url)
if ":" in hostname and not hostname.startswith("["):
hostname = f"[{hostname}]" # Add brackets around IPv6 addresses
ALLOWED_HOSTS.append(hostname)
return url
@@ -701,7 +690,7 @@ logging.config.dictConfig(LOGGING)
# https://docs.celeryq.dev/en/stable/userguide/configuration.html
# Tasks outside tasks.py must be loaded even when Django skips checks
CELERY_IMPORTS = ("documents.bulk_edit", "documents.bulk_operations")
CELERY_IMPORTS = ("documents.bulk_edit",)
CELERY_BROKER_URL = _CELERY_REDIS_URL
CELERY_RESULT_BACKEND = _CELERY_REDIS_URL
-11
View File
@@ -165,17 +165,6 @@ def parse_beat_schedule() -> dict:
"expires": 23.0 * 60.0 * 60.0,
},
},
{
"name": "Clear expired sessions",
"env_key": "PAPERLESS_SESSION_CLEANUP_TASK_CRON",
# Default daily at 03:00
"env_default": "0 3 * * *",
"task": "documents.tasks.clear_expired_sessions",
"options": {
# 1 hour before default schedule sends again
"expires": 23.0 * 60.0 * 60.0,
},
},
]
for task in tasks:
# Either get the environment setting or use the default
@@ -243,34 +243,6 @@ class TestParserSettingsFromDb(DirectoriesMixin, FileSystemAssertsMixin, TestCas
"UseDeviceIndependentColor",
)
def test_ocr_user_args_cannot_override_reserved(self) -> None:
"""
GIVEN:
- User arguments which attempt to override arguments the parser
manages itself, alongside a legitimate argument
WHEN:
- OCR parameters are constructed
THEN:
- Reserved arguments keep the parser supplied values
- The legitimate argument is still applied
"""
instance = ApplicationConfiguration.objects.all().first()
assert instance is not None
instance.user_args = {
"sidecar": "/tmp/evil.py",
"output_file": "/tmp/out.pdf",
"input_file_or_options": "/etc/passwd",
"unpaper_args": "--pre-rotate 90",
}
instance.save()
params = self.get_params()
self.assertEqual(params["sidecar"], "sidecar.txt")
self.assertEqual(params["output_file"], "output.pdf")
self.assertEqual(params["input_file_or_options"], "input.pdf")
self.assertEqual(params["unpaper_args"], "--pre-rotate 90")
def test_ocr_user_args(self) -> None:
"""
GIVEN:
@@ -18,7 +18,6 @@ from ocrmypdf import SubprocessOutputError
from PIL import Image
from documents.parsers import ParseError
from documents.parsers import get_default_thumbnail
from documents.parsers import rasterize_pdf_page_to_png
from paperless.models import ModeChoices
from paperless.parsers import ParserProtocol
@@ -271,57 +270,6 @@ class TestDpiHelpers:
class TestGetThumbnail:
@pytest.mark.parametrize(
("filename", "mime_type"),
[
pytest.param("simple.jpg", "image/jpeg", id="jpeg"),
pytest.param("simple.png", "image/png", id="png"),
pytest.param("simple.tif", "image/tiff", id="tiff"),
pytest.param("simple.gif", "image/gif", id="gif"),
pytest.param("simple.bmp", "image/bmp", id="bmp"),
pytest.param("document.webp", "image/webp", id="webp"),
pytest.param("simple.heic", "image/heic", id="heic"),
],
)
def test_image_without_archive_has_image_thumbnail(
self,
mocker: MockerFixture,
tesseract_parser: RasterisedDocumentParser,
tesseract_samples_dir: Path,
filename: str,
mime_type: str,
) -> None:
original = tesseract_samples_dir / filename
render_pdf = mocker.patch("documents.parsers.rasterize_pdf_page_to_png")
thumb = tesseract_parser.get_thumbnail(original, mime_type)
render_pdf.assert_not_called()
assert thumb.read_bytes() != get_default_thumbnail().read_bytes()
with Image.open(thumb) as im:
assert im.format == "WEBP"
assert 0 < im.width <= 500
assert 0 < im.height <= 5000
def test_image_with_archive_uses_pdf_thumbnail(
self,
mocker: MockerFixture,
tesseract_parser: RasterisedDocumentParser,
tesseract_samples_dir: Path,
simple_digital_pdf_file: Path,
) -> None:
tesseract_parser.archive_path = simple_digital_pdf_file
target = "paperless.parsers.tesseract.make_thumbnail_from_pdf"
make_thumbnail = mocker.patch(target)
original = tesseract_samples_dir / "simple.jpg"
tesseract_parser.get_thumbnail(original, "image/jpeg")
make_thumbnail.assert_called_once_with(
simple_digital_pdf_file,
tesseract_parser.tempdir,
)
def test_thumbnail_is_file(
self,
tesseract_parser: RasterisedDocumentParser,
@@ -182,7 +182,6 @@ def make_expected_schedule(
workflow_expire = 59.0 * 60.0
llm_index_expire = 23.0 * 60.0 * 60.0
share_link_cleanup_expire = 23.0 * 60.0 * 60.0
session_cleanup_expire = 23.0 * 60.0 * 60.0
schedule: dict[str, Any] = {
"Check all e-mail accounts": {
@@ -249,14 +248,6 @@ def make_expected_schedule(
"headers": {"trigger_source": "scheduled"},
},
},
"Clear expired sessions": {
"task": "documents.tasks.clear_expired_sessions",
"schedule": crontab(minute=0, hour="3"),
"options": {
"expires": session_cleanup_expire,
"headers": {"trigger_source": "scheduled"},
},
},
}
overrides = overrides or {}
@@ -307,7 +298,6 @@ class TestParseBeatSchedule:
"PAPERLESS_WORKFLOW_SCHEDULED_TASK_CRON": "disable",
"PAPERLESS_LLM_INDEX_TASK_CRON": "disable",
"PAPERLESS_SHARE_LINK_BUNDLE_CLEANUP_CRON": "disable",
"PAPERLESS_SESSION_CLEANUP_TASK_CRON": "disable",
},
{},
id="all-disabled",
+4 -89
View File
@@ -3,14 +3,9 @@ from unittest import TestCase
from unittest import mock
import pytest
from django.core.exceptions import DisallowedHost
from django.core.exceptions import ImproperlyConfigured
from django.http import HttpRequest
from django.test import override_settings
import paperless.settings as paperless_settings
from paperless.settings import _get_allauth_trusted_proxy_count
from paperless.settings import _get_allowed_hosts
from paperless.settings import _get_classifier_language_setting
from paperless.settings import _get_search_language_setting
from paperless.settings import _parse_paperless_url
@@ -173,88 +168,8 @@ class TestPaperlessURLSettings(TestCase):
self.assertIn(url, settings.CORS_ALLOWED_ORIGINS)
@pytest.mark.parametrize(
("url", "allowed_hosts", "expected", "accepted_host", "rejected_host"),
[
(None, None, ["*"], "other.example.com", None),
(None, "", ["localhost"], "localhost:8000", "other.example.com"),
(
None,
"lan.example.com",
["lan.example.com", "localhost"],
"lan.example.com",
"other.example.com",
),
(None, "*", ["*"], "other.example.com", None),
(
"https://example.com:8443",
None,
["localhost", "example.com"],
"example.com:8443",
"other.example.com",
),
(
"https://example.com.",
"",
["localhost", "example.com"],
"example.com.",
"other.example.com",
),
(
"http://[::1]:8000",
None,
["localhost", "[::1]"],
"[::1]:8000",
"other.example.com",
),
("https://example.com", "*", ["*", "example.com"], "other.example.com", None),
(
"https://example.com",
"lan.example.com",
["lan.example.com", "localhost", "example.com"],
"lan.example.com",
"other.example.com",
),
],
)
def test_allowed_hosts_at_startup(
monkeypatch: pytest.MonkeyPatch,
url: str | None,
allowed_hosts: str | None,
expected: list[str],
accepted_host: str,
rejected_host: str | None,
) -> None:
for name, value in (
("PAPERLESS_URL", url),
("PAPERLESS_ALLOWED_HOSTS", allowed_hosts),
):
if value is None:
monkeypatch.delenv(name, raising=False)
else:
monkeypatch.setenv(name, value)
monkeypatch.setattr(paperless_settings, "ALLOWED_HOSTS", _get_allowed_hosts())
monkeypatch.setattr(paperless_settings, "CSRF_TRUSTED_ORIGINS", [])
monkeypatch.setattr(paperless_settings, "CORS_ALLOWED_ORIGINS", [])
assert _parse_paperless_url() == url
assert expected == paperless_settings.ALLOWED_HOSTS
request = HttpRequest()
with override_settings(ALLOWED_HOSTS=expected):
request.META["HTTP_HOST"] = accepted_host
assert request.get_host() == accepted_host
if rejected_host:
request.META["HTTP_HOST"] = rejected_host
with pytest.raises(DisallowedHost):
request.get_host()
@pytest.mark.parametrize(
"url",
["example.com", "https://", "/just/a/path", "https://*", "https://.example.com"],
)
def test_paperless_url_without_valid_hostname_is_rejected(
@pytest.mark.parametrize("url", ["example.com", "https://", "/just/a/path"])
def test_paperless_url_without_hostname_is_rejected(
monkeypatch: pytest.MonkeyPatch,
url: str,
) -> None:
@@ -264,9 +179,9 @@ def test_paperless_url_without_valid_hostname_is_rejected(
WHEN:
- The URL is parsed
THEN:
- A configuration error is raised for missing or wildcard hostnames
- A configuration error is raised instead of adding None to ALLOWED_HOSTS
"""
monkeypatch.setenv("PAPERLESS_URL", url)
with pytest.raises(ImproperlyConfigured, match="no valid hostname"):
with pytest.raises(ImproperlyConfigured, match="no hostname"):
_parse_paperless_url()
+2 -2
View File
@@ -32,13 +32,13 @@ from rest_framework.fields import BooleanField
from rest_framework.filters import OrderingFilter
from rest_framework.generics import GenericAPIView
from rest_framework.pagination import PageNumberPagination
from rest_framework.permissions import DjangoModelPermissions
from rest_framework.permissions import IsAuthenticated
from rest_framework.response import Response
from rest_framework.throttling import ScopedRateThrottle
from rest_framework.viewsets import ModelViewSet
from documents.models import PaperlessTask
from documents.permissions import PaperlessModelPermissions
from documents.permissions import PaperlessObjectPermissions
from documents.tasks import llmindex_index
from paperless.filters import GroupFilterSet
@@ -414,7 +414,7 @@ class ApplicationConfigurationViewSet(ModelViewSet[ApplicationConfiguration]):
queryset = ApplicationConfiguration.objects
serializer_class = ApplicationConfigurationSerializer
permission_classes = (IsAuthenticated, PaperlessModelPermissions)
permission_classes = (IsAuthenticated, DjangoModelPermissions)
@extend_schema(exclude=True)
def create(self, request, *args, **kwargs):
+1 -1
View File
@@ -22,8 +22,8 @@ logger = logging.getLogger("paperless_ai.chat")
CHAT_METADATA_DELIMITER = "\n\n__PAPERLESS_CHAT_METADATA__"
CHAT_ERROR_MESSAGE = "Sorry, something went wrong while generating a response."
CHAT_NO_CONTENT_MESSAGE = "Sorry, I couldn't find any content to answer your question."
MAX_CHAT_REFERENCES = 3
CHAT_RETRIEVER_TOP_K = 5
MAX_CHAT_REFERENCES = CHAT_RETRIEVER_TOP_K
def _build_chat_prompt(output_language: str | None) -> str:
+5 -20
View File
@@ -1,5 +1,4 @@
import re
from contextlib import suppress
from typing import TYPE_CHECKING
from django.conf import settings
@@ -21,17 +20,7 @@ OCR_LEADER_REGEX = re.compile(r"[._\-\u00b7]{4,}")
HORIZONTAL_WHITESPACE_REGEX = re.compile(r"[ \t\u00a0]+")
def get_embedding_model(
config: AIConfig,
*,
refresh: bool = False,
) -> "BaseEmbedding":
"""
Build the embedding model for the configured backend.
For Hugging Face, a cached model loads offline by default, saving seconds per
call. Pass ``refresh=True`` to check the Hub for a newer model revision.
"""
def get_embedding_model(config: AIConfig) -> "BaseEmbedding":
match config.llm_embedding_backend:
case LLMEmbeddingBackend.OPENAI_LIKE:
from llama_index.embeddings.openai_like import OpenAILikeEmbedding
@@ -63,15 +52,11 @@ def get_embedding_model(
case LLMEmbeddingBackend.HUGGINGFACE:
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
hf_kwargs = {
"model_name": config.llm_embedding_model
return HuggingFaceEmbedding(
model_name=config.llm_embedding_model
or "sentence-transformers/all-MiniLM-L6-v2",
"cache_folder": str(settings.DATA_DIR / "hf_cache"),
}
if not refresh:
with suppress(OSError):
return HuggingFaceEmbedding(**hf_kwargs, local_files_only=True)
return HuggingFaceEmbedding(**hf_kwargs)
cache_folder=str(settings.DATA_DIR / "hf_cache"),
)
case LLMEmbeddingBackend.OLLAMA:
from llama_index.embeddings.ollama import OllamaEmbedding
from ollama import AsyncClient
+1 -1
View File
@@ -470,7 +470,7 @@ def update_llm_index(
logger.warning("No documents found to index.")
chunk_size = config.llm_embedding_chunk_size
embed_model = get_embedding_model(config, refresh=rebuild)
embed_model = get_embedding_model(config)
with write_store(embed_model_name=model_name) as store:
if rebuild or not store.table_exists():
@@ -212,10 +212,6 @@ def test_update_llm_index(
ai_config.assert_called_once()
build_document_node.assert_called_once_with(real_document, chunk_size=512)
indexing.get_embedding_model.assert_called_once_with(
mock_config,
refresh=True,
)
@pytest.mark.django_db
@@ -340,8 +336,6 @@ def test_update_llm_index_partial_update(
indexing.update_llm_index(rebuild=False)
assert indexing.get_embedding_model.call_args.kwargs == {"refresh": False}
with indexing.get_vector_store() as store:
assert store.table_exists(), (
"Expected the vector store table to exist after incremental update"
-36
View File
@@ -1,6 +1,5 @@
from unittest.mock import ANY
from unittest.mock import MagicMock
from unittest.mock import call
from unittest.mock import patch
import pytest
@@ -168,45 +167,10 @@ def test_get_embedding_model_huggingface(mock_ai_config):
MockHuggingFaceEmbedding.assert_called_once_with(
model_name="sentence-transformers/all-MiniLM-L6-v2",
cache_folder=str(settings.DATA_DIR / "hf_cache"),
local_files_only=True,
)
assert model == MockHuggingFaceEmbedding.return_value
def test_get_embedding_model_huggingface_downloads_when_not_cached(mock_ai_config):
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.HUGGINGFACE
mock_ai_config.return_value.llm_embedding_model = "some/model"
downloaded = MagicMock()
with patch(
"llama_index.embeddings.huggingface.HuggingFaceEmbedding",
side_effect=[OSError("not cached"), downloaded],
) as MockHuggingFaceEmbedding:
model = get_embedding_model(mock_ai_config.return_value)
cache_folder = str(settings.DATA_DIR / "hf_cache")
assert MockHuggingFaceEmbedding.call_args_list == [
call(model_name="some/model", cache_folder=cache_folder, local_files_only=True),
call(model_name="some/model", cache_folder=cache_folder),
]
assert model is downloaded
def test_get_embedding_model_huggingface_refresh_checks_the_hub(mock_ai_config):
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.HUGGINGFACE
mock_ai_config.return_value.llm_embedding_model = "some/model"
with patch(
"llama_index.embeddings.huggingface.HuggingFaceEmbedding",
) as MockHuggingFaceEmbedding:
get_embedding_model(mock_ai_config.return_value, refresh=True)
MockHuggingFaceEmbedding.assert_called_once_with(
model_name="some/model",
cache_folder=str(settings.DATA_DIR / "hf_cache"),
)
def test_get_embedding_model_ollama(mock_ai_config):
mock_ai_config.return_value.llm_embedding_backend = LLMEmbeddingBackend.OLLAMA
mock_ai_config.return_value.llm_embedding_model = "embeddinggemma"
+1 -7
View File
@@ -1159,13 +1159,7 @@ class MailAccountHandler(LoggingMixin):
new_headers += message.obj._headers
message.obj._headers = new_headers
# RFC 5322 requires CRLF; the default compat32 policy writes bare LF,
# which Outlook misreads as literal "=" in quoted-printable soft breaks
f.write(
message.obj.as_bytes(
policy=message.obj.policy.clone(linesep="\r\n"),
),
)
f.write(message.obj.as_bytes())
correspondent = self._get_correspondent(message, rule)
Generated
+33 -33
View File
@@ -3149,11 +3149,11 @@ mariadb = [
]
postgres = [
{ name = "psycopg", extra = ["c", "pool"] },
{ name = "psycopg-c", version = "3.3.8", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12' or python_full_version == '3.13.*' or python_full_version >= '3.15' or (platform_machine != 'aarch64' and platform_machine != 'x86_64') or sys_platform != 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl" }, marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl" }, marker = "python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.12' or python_full_version == '3.13.*' or python_full_version >= '3.15' or (platform_machine != 'aarch64' and platform_machine != 'x86_64') or sys_platform != 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" }, marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" }, marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl" }, marker = "python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-pool" },
]
webserver = [
@@ -3272,12 +3272,12 @@ requires-dist = [
{ name = "openai", specifier = ">=2.48" },
{ name = "pathvalidate", specifier = "~=3.3.1" },
{ name = "pdf2image", specifier = "~=1.17.0" },
{ name = "psycopg", extras = ["c", "pool"], marker = "extra == 'postgres'", specifier = "==3.3.8" },
{ name = "psycopg-c", marker = "python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl" },
{ name = "psycopg-c", marker = "(python_full_version < '3.12' and extra == 'postgres') or (python_full_version == '3.13.*' and extra == 'postgres') or (python_full_version >= '3.15' and extra == 'postgres') or (platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'postgres') or (sys_platform != 'linux' and extra == 'postgres')", specifier = "==3.3.8" },
{ name = "psycopg", extras = ["c", "pool"], marker = "extra == 'postgres'", specifier = "==3.3.4" },
{ name = "psycopg-c", marker = "python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" },
{ name = "psycopg-c", marker = "python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux' and extra == 'postgres'", url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl" },
{ name = "psycopg-c", marker = "(python_full_version < '3.12' and extra == 'postgres') or (python_full_version == '3.13.*' and extra == 'postgres') or (python_full_version >= '3.15' and extra == 'postgres') or (platform_machine != 'aarch64' and platform_machine != 'x86_64' and extra == 'postgres') or (sys_platform != 'linux' and extra == 'postgres')", specifier = "==3.3.4" },
{ name = "psycopg-pool", marker = "extra == 'postgres'", specifier = "==3.3.3" },
{ name = "python-dateutil", specifier = "~=2.9.0" },
{ name = "python-dotenv", specifier = "~=1.2.1" },
@@ -3723,23 +3723,23 @@ wheels = [
[[package]]
name = "psycopg"
version = "3.3.8"
version = "3.3.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/11/c8/3bd7e09770a5829eb26667df5e49f084d78bd106d5949ade9447288a8e18/psycopg-3.3.8.tar.gz", hash = "sha256:86c135a9a6f5e98404975c5787e620bf436a7807e088baebd1b20fa48d3b4b4f", size = 168902, upload-time = "2026-11-04T09:44:26.142Z" }
sdist = { url = "https://files.pythonhosted.org/packages/db/2f/cb91e5502ec9de1de6f1b76cfbf69531932725361168bb06963620c77e2e/psycopg-3.3.4.tar.gz", hash = "sha256:e21207764952cff81b6b8bdacad9a3939f2793367fdac2987b3aac36a651b5bc", size = 165799, upload-time = "2026-05-01T23:31:55.179Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/7b/5c/50a0b0796ccb69b11934f7c5758051c08019bda2fafb417501109fffd569/psycopg-3.3.8-py3-none-any.whl", hash = "sha256:ca50af6072ba9ea6866f33021975230c2555d7bfdb9d8f94ca3af5c3d1bf2167", size = 216209, upload-time = "2026-11-04T09:37:35.491Z" },
{ url = "https://files.pythonhosted.org/packages/5c/e0/7b3dee031daae7743609ce3c746565d4a3ed7c2c186479eb48e34e838c64/psycopg-3.3.4-py3-none-any.whl", hash = "sha256:b6bbc25ccf05c8fad3b061d9db2ef0909a555171b84b07f29458a447253d679a", size = 213001, upload-time = "2026-05-01T23:20:50.816Z" },
]
[package.optional-dependencies]
c = [
{ name = "psycopg-c", version = "3.3.8", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.12' and implementation_name != 'pypy') or (python_full_version == '3.13.*' and implementation_name != 'pypy') or (python_full_version >= '3.15' and implementation_name != 'pypy') or (implementation_name != 'pypy' and platform_machine != 'aarch64' and platform_machine != 'x86_64') or (implementation_name != 'pypy' and sys_platform != 'linux')" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl" }, marker = "python_full_version == '3.14.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.8", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl" }, marker = "python_full_version == '3.14.*' and implementation_name != 'pypy' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { registry = "https://pypi.org/simple" }, marker = "(python_full_version < '3.12' and implementation_name != 'pypy') or (python_full_version == '3.13.*' and implementation_name != 'pypy') or (python_full_version >= '3.15' and implementation_name != 'pypy') or (implementation_name != 'pypy' and platform_machine != 'aarch64' and platform_machine != 'x86_64') or (implementation_name != 'pypy' and sys_platform != 'linux')" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" }, marker = "python_full_version == '3.12.*' and implementation_name != 'pypy' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" }, marker = "python_full_version == '3.14.*' and implementation_name != 'pypy' and platform_machine == 'aarch64' and sys_platform == 'linux'" },
{ name = "psycopg-c", version = "3.3.4", source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl" }, marker = "python_full_version == '3.14.*' and implementation_name != 'pypy' and platform_machine == 'x86_64' and sys_platform == 'linux'" },
]
pool = [
{ name = "psycopg-pool" },
@@ -3747,7 +3747,7 @@ pool = [
[[package]]
name = "psycopg-c"
version = "3.3.8"
version = "3.3.4"
source = { registry = "https://pypi.org/simple" }
resolution-markers = [
"python_full_version >= '3.15' and sys_platform == 'darwin'",
@@ -3757,50 +3757,50 @@ resolution-markers = [
"python_full_version < '3.12' and sys_platform == 'darwin'",
"python_full_version < '3.12' and sys_platform == 'linux'",
]
sdist = { url = "https://files.pythonhosted.org/packages/5b/23/0074f50899d773aae857af496745cd898da868f8f2d9415591d51c391cec/psycopg_c-3.3.8.tar.gz", hash = "sha256:af81a99b02600e6c2cdf468509c04bb8208500ea60d63f70616e32ea9cb2758f", size = 700571, upload-time = "2026-11-04T09:44:27.694Z" }
sdist = { url = "https://files.pythonhosted.org/packages/21/7c/c08364f2eab2913e4068b3b955d963e7a3491986a85429990969525def30/psycopg_c-3.3.4.tar.gz", hash = "sha256:ed8106128b2d04359c185fc9641b4409abfce4d0b6fb1d1ff6800646e27f1a22", size = 647111, upload-time = "2026-05-01T23:31:58.032Z" }
[[package]]
name = "psycopg-c"
version = "3.3.8"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl" }
version = "3.3.4"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl" }
resolution-markers = [
"python_full_version == '3.12.*' and platform_machine == 'aarch64' and sys_platform == 'linux'",
]
wheels = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_aarch64.whl", hash = "sha256:be9f81019ec26eb09b8852ec9cb156a41d91067bbc2093cee93daabc935068e8" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_aarch64.whl", hash = "sha256:44ea0e800dc2126a0d3ce23fac7b96b64db132ed67df3e9ceb011965351ae8dc" },
]
[[package]]
name = "psycopg-c"
version = "3.3.8"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl" }
version = "3.3.4"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl" }
resolution-markers = [
"python_full_version == '3.12.*' and platform_machine == 'x86_64' and sys_platform == 'linux'",
]
wheels = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp312-cp312-manylinux_2_38_x86_64.whl", hash = "sha256:72a24b31600ec991cb75852038de012054b6031b355c95ccfc38680e439559cc" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp312-cp312-linux_x86_64.whl", hash = "sha256:c66169fef7be6b46e701dd361036bd88e346e6b8b15dd25528287f807654f097" },
]
[[package]]
name = "psycopg-c"
version = "3.3.8"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl" }
version = "3.3.4"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl" }
resolution-markers = [
"python_full_version == '3.14.*' and platform_machine == 'aarch64' and sys_platform == 'linux'",
]
wheels = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_aarch64.whl", hash = "sha256:dc22c1b4eeee9a2a5e20a8ffbd192e320a2da6fdef5a72f7fa76ed9792de4353" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_aarch64.whl", hash = "sha256:00d1cc7b82899046acd68e11c801bbb94d109313ab02333f76c2b59dd531cdc1" },
]
[[package]]
name = "psycopg-c"
version = "3.3.8"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl" }
version = "3.3.4"
source = { url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl" }
resolution-markers = [
"python_full_version == '3.14.*' and platform_machine == 'x86_64' and sys_platform == 'linux'",
]
wheels = [
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.8/psycopg_c-3.3.8-cp314-cp314-manylinux_2_38_x86_64.whl", hash = "sha256:d6298166033ad1024d2ba6d0805352ba7042ccd7619771a0c164d1e370cf7976" },
{ url = "https://github.com/paperless-ngx/builder/releases/download/psycopg-trixie-3.3.4/psycopg_c-3.3.4-cp314-cp314-linux_x86_64.whl", hash = "sha256:d02a564c6a3e2c1957b3b5d2dff77e34a274f8551f6bdbcef47523eba59a627a" },
]
[[package]]