Compare commits

...

68 Commits

Author SHA1 Message Date
shamoon
0e0407fc40 empty commit to test ci 2026-03-05 16:13:18 -08:00
shamoon
afd5e4a3cd Add path-change detection and CI gate jobs 2026-03-05 15:59:06 -08:00
dependabot[bot]
a5a267fe49 Bump django-allauth from 65.14.0 to 65.14.1 (#12253)
Bumps [django-allauth](https://github.com/sponsors/pennersr) from 65.14.0 to 65.14.1.
- [Commits](https://github.com/sponsors/pennersr/commits)

---
updated-dependencies:
- dependency-name: django-allauth
  dependency-version: 65.14.1
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-05 14:32:04 -08:00
shamoon
24a2cfd957 Change: use explicit doc creation instead of clone for versions (#12226) 2026-03-04 15:57:44 -08:00
GitHub Actions
7cf2ef6398 Auto translate strings 2026-03-04 23:29:54 +00:00
shamoon
df03207eef Fix: correct doc version filename handling (#12223) 2026-03-04 23:28:07 +00:00
dependabot[bot]
fa998ecd49 Bump django from 5.2.11 to 5.2.12 (#12249)
Bumps [django](https://github.com/django/django) from 5.2.11 to 5.2.12.
- [Commits](https://github.com/django/django/compare/5.2.11...5.2.12)

---
updated-dependencies:
- dependency-name: django
  dependency-version: 5.2.12
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-04 15:16:25 -08:00
Trenton H
1e21bcd26e Breaking: Drop support for Python 3.10 (#12234) 2026-03-04 15:03:33 -08:00
Trenton H
a9cb89c633 Enhancement: Improve exporter memory efficiency (#12236)
Phase 1 -- Eliminate JSON round-trip in document exporter

Replace json.loads(serializers.serialize("json", qs)) with
serializers.serialize("python", qs) to skip the intermediate
JSON string allocation and parse step. Use DjangoJSONEncoder
in check_and_write_json() to handle native Python types
(datetime, Decimal, UUID) the Python serializer returns.

Phase 2 -- Batched QuerySet serialization in document exporter

Add serialize_queryset_batched() helper that uses QuerySet.iterator()
and itertools.islice to stream records in configurable chunks, bounding
peak memory during serialization to batch_size * avg_record_size rather
than loading the entire QuerySet at once.
2026-03-04 14:54:20 -08:00
GitHub Actions
a37e24c1ad Auto translate strings 2026-03-04 22:17:32 +00:00
shamoon
85a18e5911 Enhancement: saved view sharing (#12142) 2026-03-04 14:15:43 -08:00
GitHub Actions
ae182c459b Auto translate strings 2026-03-04 21:34:02 +00:00
shamoon
d51a118aac Merge branch 'main' into dev 2026-03-04 13:31:20 -08:00
github-actions[bot]
d6a316b1df Changelog v2.20.10 - GHA (#12247)
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
2026-03-04 11:25:44 -08:00
shamoon
8f311c4b6b Bump version to 2.20.10 2026-03-04 10:38:14 -08:00
shamoon
f25322600d Merge branch 'release/v2.20.x' 2026-03-04 10:09:01 -08:00
shamoon
615f27e6fb Fix: support string coercion in filepath jinja templates (#12244) 2026-03-04 08:32:34 -08:00
Andreas Schneider
190fc70288 Fix: use maxsplit=1 in Redis URL parsing to handle URLs with multiple colons (#12239) 2026-03-04 01:06:51 -08:00
shamoon
5b809122b5 Fix: apply ordering after annotating tag document count (#12238) 2026-03-04 00:33:13 -08:00
GitHub Actions
c623234769 Auto translate strings 2026-03-04 00:29:21 +00:00
shamoon
299dac21ee Enhancement: “live” document updates (#12141) 2026-03-04 00:27:07 +00:00
Trenton H
5498503d60 Chore: Improve user migration path (#12232)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-03-03 15:51:48 -08:00
shamoon
8b8307571a Fix: enforce path limit for db filename fields (#12235) 2026-03-03 13:19:56 -08:00
GitHub Actions
16b58c2de5 Auto translate strings 2026-03-03 19:25:03 +00:00
shamoon
c724fbb5d9 Clarify bulk edit wording with versions 2026-03-03 11:22:22 -08:00
dependabot[bot]
9c0f112e94 docker(deps): Bump astral-sh/uv (#12191)
Bumps [astral-sh/uv](https://github.com/astral-sh/uv) from 0.10.5-python3.12-trixie-slim to 0.10.7-python3.12-trixie-slim.
- [Release notes](https://github.com/astral-sh/uv/releases)
- [Changelog](https://github.com/astral-sh/uv/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/uv/compare/0.10.5...0.10.7)

---
updated-dependencies:
- dependency-name: astral-sh/uv
  dependency-version: 0.10.7-python3.12-trixie-slim
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-03 07:56:35 -08:00
Trenton H
43406f44f2 Feature: Improve the retagger output using rich (#12194) 2026-03-03 07:14:59 -08:00
shamoon
b7ca3550b1 Merge branch 'main' into dev 2026-03-02 13:45:10 -08:00
shamoon
0e97419e0e Chore: add existing logo for temporary url resolution 2026-03-02 13:43:24 -08:00
dependabot[bot]
10cb2ac183 Chore(deps): Bump the actions group across 1 directory with 6 updates (#12224)
Bumps the actions group with 6 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `7.3.0` | `7.3.1` |
| [actions/upload-artifact](https://github.com/actions/upload-artifact) | `6.0.0` | `7.0.0` |
| [actions/download-artifact](https://github.com/actions/download-artifact) | `7.0.0` | `8.0.0` |
| [github/codeql-action](https://github.com/github/codeql-action) | `4.32.3` | `4.32.5` |
| [crowdin/github-action](https://github.com/crowdin/github-action) | `2.14.0` | `2.15.0` |
| [actions/stale](https://github.com/actions/stale) | `10.1.1` | `10.2.0` |



Updates `astral-sh/setup-uv` from 7.3.0 to 7.3.1
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/v7.3.0...v7.3.1)

Updates `actions/upload-artifact` from 6.0.0 to 7.0.0
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/v6.0.0...v7.0.0)

Updates `actions/download-artifact` from 7.0.0 to 8.0.0
- [Release notes](https://github.com/actions/download-artifact/releases)
- [Commits](https://github.com/actions/download-artifact/compare/v7.0.0...v8.0.0)

Updates `github/codeql-action` from 4.32.3 to 4.32.5
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v4.32.3...v4.32.5)

Updates `crowdin/github-action` from 2.14.0 to 2.15.0
- [Release notes](https://github.com/crowdin/github-action/releases)
- [Commits](https://github.com/crowdin/github-action/compare/v2.14.0...v2.15.0)

Updates `actions/stale` from 10.1.1 to 10.2.0
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/stale/compare/v10.1.1...v10.2.0)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: 7.3.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: actions/upload-artifact
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: actions/download-artifact
  dependency-version: 8.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: actions
- dependency-name: github/codeql-action
  dependency-version: 4.32.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: actions
- dependency-name: crowdin/github-action
  dependency-version: 2.15.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
- dependency-name: actions/stale
  dependency-version: 10.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: actions
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 11:55:24 -08:00
Trenton H
1d7cd5a7ad Chore: Updates actions to the most specific version released (#12222) 2026-03-02 11:34:57 -08:00
Trenton H
e58a35d40c Feature: Transition sanity check to rich and improve output (#12182) 2026-03-02 10:53:39 -08:00
Trenton H
20a9cd40e8 Feature: Switch all indexing to use rich (#12193) 2026-03-02 10:41:09 -08:00
dependabot[bot]
b94ce85b46 Chore(deps): Bump whitenoise in the django-ecosystem group (#12192)
Bumps the django-ecosystem group with 1 update: [whitenoise](https://github.com/evansd/whitenoise).


Updates `whitenoise` from 6.11.0 to 6.12.0
- [Changelog](https://github.com/evansd/whitenoise/blob/main/docs/changelog.rst)
- [Commits](https://github.com/evansd/whitenoise/compare/6.11.0...6.12.0)

---
updated-dependencies:
- dependency-name: whitenoise
  dependency-version: 6.12.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django-ecosystem
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 10:25:11 -08:00
dependabot[bot]
484bef00c1 docker-compose(deps): Bump gotenberg/gotenberg in /docker/compose (#12190)
Bumps gotenberg/gotenberg from 8.26 to 8.27.

---
updated-dependencies:
- dependency-name: gotenberg/gotenberg
  dependency-version: '8.27'
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 10:14:48 -08:00
Trenton H
317a177537 Chore: Updates s6-overlay to 3.2.2.0 (#12189) 2026-03-02 09:00:03 -08:00
GitHub Actions
62efb4078f Auto translate strings 2026-03-02 16:23:02 +00:00
shamoon
96ac7b2336 Tweak: Ignore version docs for workflows (#12217) 2026-03-02 08:21:14 -08:00
GitHub Actions
20173a2863 Auto translate strings 2026-03-02 00:12:53 -08:00
dependabot[bot]
4e883ad70f Chore(deps): Bump zone.js from 0.16.0 to 0.16.1 in /src-ui (#12214)
Bumps [zone.js](https://github.com/angular/angular/tree/HEAD/packages/zone.js) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/packages/zone.js/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/zone.js-0.16.1/packages/zone.js)

---
updated-dependencies:
- dependency-name: zone.js
  dependency-version: 0.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 00:12:53 -08:00
dependabot[bot]
fab3c04b3f Chore(deps-dev): Bump webpack from 5.105.0 to 5.105.3 in /src-ui (#12213)
Bumps [webpack](https://github.com/webpack/webpack) from 5.105.0 to 5.105.3.
- [Release notes](https://github.com/webpack/webpack/releases)
- [Changelog](https://github.com/webpack/webpack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/webpack/webpack/compare/v5.105.0...v5.105.3)

---
updated-dependencies:
- dependency-name: webpack
  dependency-version: 5.105.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 00:12:53 -08:00
dependabot[bot]
2c87ba9bd6 Chore(deps-dev): Bump jest-preset-angular (#12211)
Bumps the frontend-jest-dependencies group in /src-ui with 1 update: [jest-preset-angular](https://github.com/thymikee/jest-preset-angular).


Updates `jest-preset-angular` from 16.0.0 to 16.1.1
- [Release notes](https://github.com/thymikee/jest-preset-angular/releases)
- [Changelog](https://github.com/thymikee/jest-preset-angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/thymikee/jest-preset-angular/compare/v16.0.0...v16.1.1)

---
updated-dependencies:
- dependency-name: jest-preset-angular
  dependency-version: 16.1.1
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-jest-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 00:12:53 -08:00
dependabot[bot]
3d843af64d Chore(deps-dev): Bump @types/node from 25.2.1 to 25.3.3 in /src-ui (#12215)
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 25.2.1 to 25.3.3.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-version: 25.3.3
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 00:12:53 -08:00
GitHub Actions
8f3dece8a7 Auto translate strings 2026-03-02 00:12:52 -08:00
shamoon
95484db71b Update paperless_mail npm packges 2026-03-02 00:12:52 -08:00
dependabot[bot]
2003fee401 Chore(deps): Bump the frontend-angular-dependencies group (#12210)
Bumps the frontend-angular-dependencies group in /src-ui with 15 updates:

| Package | From | To |
| --- | --- | --- |
| [@angular/cdk](https://github.com/angular/components) | `21.1.3` | `21.2.0` |
| [@angular/common](https://github.com/angular/angular/tree/HEAD/packages/common) | `21.1.3` | `21.2.0` |
| [@angular/compiler](https://github.com/angular/angular/tree/HEAD/packages/compiler) | `21.1.3` | `21.2.0` |
| [@angular/core](https://github.com/angular/angular/tree/HEAD/packages/core) | `21.1.3` | `21.2.0` |
| [@angular/forms](https://github.com/angular/angular/tree/HEAD/packages/forms) | `21.1.3` | `21.2.0` |
| [@angular/localize](https://github.com/angular/angular) | `21.1.3` | `21.2.0` |
| [@angular/platform-browser](https://github.com/angular/angular/tree/HEAD/packages/platform-browser) | `21.1.3` | `21.2.0` |
| [@angular/platform-browser-dynamic](https://github.com/angular/angular/tree/HEAD/packages/platform-browser-dynamic) | `21.1.3` | `21.2.0` |
| [@angular/router](https://github.com/angular/angular/tree/HEAD/packages/router) | `21.1.3` | `21.2.0` |
| [@ng-select/ng-select](https://github.com/ng-select/ng-select) | `21.2.0` | `21.4.0` |
| [@angular-devkit/core](https://github.com/angular/angular-cli) | `21.1.3` | `21.2.0` |
| [@angular-devkit/schematics](https://github.com/angular/angular-cli) | `21.1.3` | `21.2.0` |
| [@angular/build](https://github.com/angular/angular-cli) | `21.1.3` | `21.2.0` |
| [@angular/cli](https://github.com/angular/angular-cli) | `21.1.3` | `21.2.0` |
| [@angular/compiler-cli](https://github.com/angular/angular/tree/HEAD/packages/compiler-cli) | `21.1.3` | `21.2.0` |


Updates `@angular/cdk` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/components/releases)
- [Changelog](https://github.com/angular/components/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/components/compare/v21.1.3...v21.2.0)

Updates `@angular/common` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/common)

Updates `@angular/compiler` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/compiler)

Updates `@angular/core` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/core)

Updates `@angular/forms` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/forms)

Updates `@angular/localize` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/compare/v21.1.3...v21.2.0)

Updates `@angular/platform-browser` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/platform-browser)

Updates `@angular/platform-browser-dynamic` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/platform-browser-dynamic)

Updates `@angular/router` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/router)

Updates `@ng-select/ng-select` from 21.2.0 to 21.4.0
- [Release notes](https://github.com/ng-select/ng-select/releases)
- [Changelog](https://github.com/ng-select/ng-select/blob/master/CHANGELOG.md)
- [Commits](https://github.com/ng-select/ng-select/compare/v21.2.0...v21.4.0)

Updates `@angular-devkit/core` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.1.3...v21.2.0)

Updates `@angular-devkit/schematics` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.1.3...v21.2.0)

Updates `@angular/build` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.1.3...v21.2.0)

Updates `@angular/cli` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular-cli/releases)
- [Changelog](https://github.com/angular/angular-cli/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular-cli/compare/v21.1.3...v21.2.0)

Updates `@angular/compiler-cli` from 21.1.3 to 21.2.0
- [Release notes](https://github.com/angular/angular/releases)
- [Changelog](https://github.com/angular/angular/blob/main/CHANGELOG.md)
- [Commits](https://github.com/angular/angular/commits/v21.2.0/packages/compiler-cli)

---
updated-dependencies:
- dependency-name: "@angular/cdk"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/common"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/compiler"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/core"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/forms"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/localize"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/platform-browser"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/platform-browser-dynamic"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/router"
  dependency-version: 21.2.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@ng-select/ng-select"
  dependency-version: 21.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-devkit/core"
  dependency-version: 21.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular-devkit/schematics"
  dependency-version: 21.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/build"
  dependency-version: 21.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/cli"
  dependency-version: 21.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
- dependency-name: "@angular/compiler-cli"
  dependency-version: 21.2.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: frontend-angular-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-02 00:12:52 -08:00
shamoon
8e7084eba7 Bump tailwindcss to v3.4.19 2026-03-02 00:12:52 -08:00
Jan Kleine
cd2b5127db Documentation: fix version label filename placeholder
Co-Authored-By: shamoon <4887959+shamoon@users.noreply.github.com>
2026-03-02 00:12:52 -08:00
shamoon
7ff51452f0 Documentation: small note re filename vs original_filename 2026-03-01 11:45:06 -08:00
shamoon
a700928dd5 Italicize version label 2026-03-01 09:30:03 -08:00
shamoon
709bcfd30d Fix merge thing 2026-02-28 02:34:50 -08:00
GitHub Actions
dd06627e43 Auto translate strings 2026-02-28 10:34:26 +00:00
shamoon
f65807b906 Merge branch 'main' into dev
# Conflicts:
#	docs/setup.md
#	src-ui/src/app/components/manage/document-attributes/management-list/management-list.component.ts
#	src/documents/tests/test_api_documents.py
2026-02-28 02:31:20 -08:00
github-actions[bot]
a6c974589f Documentation: Add v2.20.9 changelog (#12200)
* Changelog v2.20.9 - GHA

* Update changelog for paperless-ngx 2.20.9

Added security advisory and bug fixes for version 2.20.9.

---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-02-28 02:25:53 -08:00
shamoon
47f9f642a9 Bump version to 2.20.9 2026-02-28 01:35:26 -08:00
shamoon
8bfebc3b9b Merge branch 'release/v2.20.x' 2026-02-28 01:34:33 -08:00
shamoon
c7f83212a3 Enforce on selection_data too 2026-02-28 01:27:40 -08:00
shamoon
b010f65ae7 Fix GHSA-386h-chg4-cfw9 2026-02-28 01:16:53 -08:00
shamoon
1dd3a62bc2 Fixhancement: show sequential + id version labels, fix padding (#12196) 2026-02-27 16:22:29 -08:00
Jan Kleine
0bc032a67d Development: improve test portability (#12187)
* Fix: improve test portability

* Make settings always consistent

* Make a few more tests deterministic wrt settings

* Dont pollute settings for this one

* Fix timezone issue with mail parser

* Update test_parser.py

* Uh, I guess OCR gives variants for this

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-02-27 23:24:11 +00:00
GitHub Actions
8531078a54 Auto translate strings 2026-02-27 22:39:20 +00:00
Trenton H
5988d5896b Breaking: Refactor advanced database settings to allow more user configuration (#12165) 2026-02-27 14:37:26 -08:00
shamoon
89d3a53603 Documentation: note GHSAs in changelog 2026-02-26 23:26:35 -08:00
dependabot[bot]
898dc578e5 Chore(deps): Bump the utilities-patch group across 1 directory with 11 updates (#12179)
Bumps the utilities-patch group with 11 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [concurrent-log-handler](https://github.com/Preston-Landers/concurrent-log-handler) | `0.9.28` | `0.9.29` |
| [django-soft-delete](https://github.com/san4ezy/django_softdelete) | `1.0.22` | `1.0.23` |
| [llama-index-core](https://github.com/run-llama/llama_index) | `0.14.13` | `0.14.15` |
| llama-index-llms-openai | `0.6.18` | `0.6.21` |
| llama-index-vector-stores-faiss | `0.5.2` | `0.5.3` |
| [sentence-transformers](https://github.com/huggingface/sentence-transformers) | `5.2.2` | `5.2.3` |
| [mysqlclient](https://github.com/PyMySQL/mysqlclient) | `2.2.7` | `2.2.8` |
| [zensical](https://github.com/zensical/zensical) | `0.0.21` | `0.0.24` |
| [prek](https://github.com/j178/prek) | `0.3.2` | `0.3.3` |
| [ruff](https://github.com/astral-sh/ruff) | `0.15.0` | `0.15.3` |
| [types-markdown](https://github.com/typeshed-internal/stub_uploader) | `3.10.0.20251106` | `3.10.2.20260211` |



Updates `concurrent-log-handler` from 0.9.28 to 0.9.29
- [Release notes](https://github.com/Preston-Landers/concurrent-log-handler/releases)
- [Changelog](https://github.com/Preston-Landers/concurrent-log-handler/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Preston-Landers/concurrent-log-handler/compare/0.9.28...0.9.29)

Updates `django-soft-delete` from 1.0.22 to 1.0.23
- [Changelog](https://github.com/san4ezy/django_softdelete/blob/master/CHANGELOG.md)
- [Commits](https://github.com/san4ezy/django_softdelete/commits)

Updates `llama-index-core` from 0.14.13 to 0.14.15
- [Release notes](https://github.com/run-llama/llama_index/releases)
- [Changelog](https://github.com/run-llama/llama_index/blob/main/CHANGELOG.md)
- [Commits](https://github.com/run-llama/llama_index/compare/v0.14.13...v0.14.15)

Updates `llama-index-llms-openai` from 0.6.18 to 0.6.21

Updates `llama-index-vector-stores-faiss` from 0.5.2 to 0.5.3

Updates `sentence-transformers` from 5.2.2 to 5.2.3
- [Release notes](https://github.com/huggingface/sentence-transformers/releases)
- [Commits](https://github.com/huggingface/sentence-transformers/compare/v5.2.2...v5.2.3)

Updates `mysqlclient` from 2.2.7 to 2.2.8
- [Release notes](https://github.com/PyMySQL/mysqlclient/releases)
- [Changelog](https://github.com/PyMySQL/mysqlclient/blob/main/HISTORY.rst)
- [Commits](https://github.com/PyMySQL/mysqlclient/compare/v2.2.7...v2.2.8)

Updates `zensical` from 0.0.21 to 0.0.24
- [Release notes](https://github.com/zensical/zensical/releases)
- [Commits](https://github.com/zensical/zensical/compare/v0.0.21...v0.0.24)

Updates `prek` from 0.3.2 to 0.3.3
- [Release notes](https://github.com/j178/prek/releases)
- [Changelog](https://github.com/j178/prek/blob/master/CHANGELOG.md)
- [Commits](https://github.com/j178/prek/compare/v0.3.2...v0.3.3)

Updates `ruff` from 0.15.0 to 0.15.3
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.15.0...0.15.3)

Updates `types-markdown` from 3.10.0.20251106 to 3.10.2.20260211
- [Commits](https://github.com/typeshed-internal/stub_uploader/commits)

---
updated-dependencies:
- dependency-name: concurrent-log-handler
  dependency-version: 0.9.29
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: django-soft-delete
  dependency-version: 1.0.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: llama-index-core
  dependency-version: 0.14.15
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: llama-index-llms-openai
  dependency-version: 0.6.21
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: llama-index-vector-stores-faiss
  dependency-version: 0.5.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: sentence-transformers
  dependency-version: 5.2.3
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: mysqlclient
  dependency-version: 2.2.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: zensical
  dependency-version: 0.0.24
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: prek
  dependency-version: 0.3.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: ruff
  dependency-version: 0.15.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
- dependency-name: types-markdown
  dependency-version: 3.10.2.20260211
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: utilities-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-02-26 14:03:42 -08:00
Trenton H
c30ee1ec03 Feature: Switch progress bar library to rich (#12169) 2026-02-26 12:20:21 -08:00
shamoon
9601b3d597 Fixhancement: config option reset (#12176) 2026-02-26 10:03:54 -08:00
shamoon
13e07844fe Fix: separate displayed and API collection sizes for tags (#12170) 2026-02-25 17:25:36 -08:00
shamoon
be82fcb70a Documentation: docs cleanup (#12158) 2026-02-24 15:10:38 -08:00
157 changed files with 10918 additions and 6058 deletions

View File

@@ -14,10 +14,6 @@ component_management:
# https://docs.codecov.com/docs/carryforward-flags
flags:
# Backend Python versions
backend-python-3.10:
paths:
- src/**
carryforward: true
backend-python-3.11:
paths:
- src/**
@@ -26,6 +22,14 @@ flags:
paths:
- src/**
carryforward: true
backend-python-3.13:
paths:
- src/**
carryforward: true
backend-python-3.14:
paths:
- src/**
carryforward: true
# Frontend (shards merge into single flag)
frontend-node-24.x:
paths:
@@ -41,9 +45,10 @@ coverage:
project:
backend:
flags:
- backend-python-3.10
- backend-python-3.11
- backend-python-3.12
- backend-python-3.13
- backend-python-3.14
paths:
- src/**
# https://docs.codecov.com/docs/commit-status#threshold
@@ -59,9 +64,10 @@ coverage:
patch:
backend:
flags:
- backend-python-3.10
- backend-python-3.11
- backend-python-3.12
- backend-python-3.13
- backend-python-3.14
paths:
- src/**
target: 100%

View File

@@ -39,3 +39,6 @@ max_line_length = off
[Dockerfile*]
indent_style = space
[*.toml]
indent_style = space

View File

@@ -3,21 +3,9 @@ on:
push:
branches-ignore:
- 'translations**'
paths:
- 'src/**'
- 'pyproject.toml'
- 'uv.lock'
- 'docker/compose/docker-compose.ci-test.yml'
- '.github/workflows/ci-backend.yml'
pull_request:
branches-ignore:
- 'translations**'
paths:
- 'src/**'
- 'pyproject.toml'
- 'uv.lock'
- 'docker/compose/docker-compose.ci-test.yml'
- '.github/workflows/ci-backend.yml'
workflow_dispatch:
concurrency:
group: backend-${{ github.event.pull_request.number || github.ref }}
@@ -26,27 +14,55 @@ env:
DEFAULT_UV_VERSION: "0.10.x"
NLTK_DATA: "/usr/share/nltk_data"
jobs:
changes:
name: Detect Backend Changes
runs-on: ubuntu-24.04
outputs:
backend_changed: ${{ steps.manual.outputs.backend || steps.filter.outputs.backend || 'false' }}
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
- name: Mark backend changes for manual runs
id: manual
if: github.event_name == 'workflow_dispatch'
run: echo "backend=true" >> "$GITHUB_OUTPUT"
- name: Detect backend changes
id: filter
if: github.event_name != 'workflow_dispatch'
uses: dorny/paths-filter@v3.0.2
with:
filters: |
backend:
- 'src/**'
- 'pyproject.toml'
- 'uv.lock'
- 'docker/compose/docker-compose.ci-test.yml'
- '.github/workflows/ci-backend.yml'
test:
needs: changes
if: needs.changes.outputs.backend_changed == 'true'
name: "Python ${{ matrix.python-version }}"
runs-on: ubuntu-24.04
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12']
python-version: ['3.11', '3.12', '3.13', '3.14']
fail-fast: false
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Start containers
run: |
docker compose --file docker/compose/docker-compose.ci-test.yml pull --quiet
docker compose --file docker/compose/docker-compose.ci-test.yml up --detach
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6
uses: actions/setup-python@v6.2.0
with:
python-version: "${{ matrix.python-version }}"
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@v7.3.1
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -83,13 +99,13 @@ jobs:
pytest
- name: Upload test results to Codecov
if: always()
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v5.5.2
with:
flags: backend-python-${{ matrix.python-version }}
files: junit.xml
report_type: test_results
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v5.5.2
with:
flags: backend-python-${{ matrix.python-version }}
files: coverage.xml
@@ -100,20 +116,22 @@ jobs:
docker compose --file docker/compose/docker-compose.ci-test.yml logs
docker compose --file docker/compose/docker-compose.ci-test.yml down
typing:
needs: changes
if: needs.changes.outputs.backend_changed == 'true'
name: Check project typing
runs-on: ubuntu-24.04
env:
DEFAULT_PYTHON: "3.12"
steps:
- name: Checkout
uses: actions/checkout@v6.0.1
uses: actions/checkout@v6.0.2
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6.2.0
with:
python-version: "${{ env.DEFAULT_PYTHON }}"
- name: Install uv
uses: astral-sh/setup-uv@v7.2.1
uses: astral-sh/setup-uv@v7.3.1
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -150,3 +168,27 @@ jobs:
--show-error-codes \
--warn-unused-configs \
src/ | uv run mypy-baseline filter
gate:
name: Backend CI Gate
needs: [changes, test, typing]
if: always()
runs-on: ubuntu-24.04
steps:
- name: Validate backend workflow results
run: |
if [[ "${{ needs.changes.outputs.backend_changed }}" != "true" ]]; then
echo "No backend-relevant changes detected."
exit 0
fi
if [[ "${{ needs.test.result }}" != "success" ]]; then
echo "::error::Backend test job result: ${{ needs.test.result }}"
exit 1
fi
if [[ "${{ needs.typing.result }}" != "success" ]]; then
echo "::error::Backend typing job result: ${{ needs.typing.result }}"
exit 1
fi
echo "Backend checks passed."

View File

@@ -41,7 +41,7 @@ jobs:
ref-name: ${{ steps.ref.outputs.name }}
steps:
- name: Checkout
uses: actions/checkout@v6.0.1
uses: actions/checkout@v6.0.2
- name: Determine ref name
id: ref
run: |
@@ -130,7 +130,7 @@ jobs:
type=semver,pattern={{major}}.{{minor}}
- name: Build and push by digest
id: build
uses: docker/build-push-action@v6.18.0
uses: docker/build-push-action@v6.19.2
with:
context: .
file: ./Dockerfile
@@ -152,7 +152,7 @@ jobs:
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
if: steps.check-push.outputs.should-push == 'true'
uses: actions/upload-artifact@v6.0.0
uses: actions/upload-artifact@v7.0.0
with:
name: digests-${{ matrix.arch }}
path: /tmp/digests/*
@@ -168,7 +168,7 @@ jobs:
packages: write
steps:
- name: Download digests
uses: actions/download-artifact@v7.0.0
uses: actions/download-artifact@v8.0.0
with:
path: /tmp/digests
pattern: digests-*

View File

@@ -4,19 +4,7 @@ on:
branches:
- main
- dev
paths:
- 'docs/**'
- 'zensical.toml'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/ci-docs.yml'
pull_request:
paths:
- 'docs/**'
- 'zensical.toml'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/ci-docs.yml'
workflow_dispatch:
concurrency:
group: docs-${{ github.event.pull_request.number || github.ref }}
@@ -29,20 +17,48 @@ env:
DEFAULT_UV_VERSION: "0.10.x"
DEFAULT_PYTHON_VERSION: "3.12"
jobs:
changes:
name: Detect Docs Changes
runs-on: ubuntu-24.04
outputs:
docs_changed: ${{ steps.manual.outputs.docs || steps.filter.outputs.docs || 'false' }}
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
- name: Mark docs changes for manual runs
id: manual
if: github.event_name == 'workflow_dispatch'
run: echo "docs=true" >> "$GITHUB_OUTPUT"
- name: Detect docs changes
id: filter
if: github.event_name != 'workflow_dispatch'
uses: dorny/paths-filter@v3.0.2
with:
filters: |
docs:
- 'docs/**'
- 'zensical.toml'
- 'pyproject.toml'
- 'uv.lock'
- '.github/workflows/ci-docs.yml'
build:
needs: changes
if: needs.changes.outputs.docs_changed == 'true'
name: Build Documentation
runs-on: ubuntu-24.04
steps:
- uses: actions/configure-pages@v5
- uses: actions/configure-pages@v5.0.0
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6
uses: actions/setup-python@v6.2.0
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@v7.3.1
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -58,21 +74,40 @@ jobs:
--frozen \
zensical build --clean
- name: Upload GitHub Pages artifact
uses: actions/upload-pages-artifact@v4
uses: actions/upload-pages-artifact@v4.0.0
with:
path: site
name: github-pages-${{ github.run_id }}-${{ github.run_attempt }}
deploy:
name: Deploy Documentation
needs: build
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: [changes, build]
if: github.event_name == 'push' && github.ref == 'refs/heads/main' && needs.changes.outputs.docs_changed == 'true'
runs-on: ubuntu-24.04
environment:
name: github-pages
url: ${{ steps.deployment.outputs.page_url }}
steps:
- name: Deploy GitHub Pages
uses: actions/deploy-pages@v4
uses: actions/deploy-pages@v4.0.5
id: deployment
with:
artifact_name: github-pages-${{ github.run_id }}-${{ github.run_attempt }}
gate:
name: Docs CI Gate
needs: [changes, build]
if: always()
runs-on: ubuntu-24.04
steps:
- name: Validate docs workflow results
run: |
if [[ "${{ needs.changes.outputs.docs_changed }}" != "true" ]]; then
echo "No docs-relevant changes detected."
exit 0
fi
if [[ "${{ needs.build.result }}" != "success" ]]; then
echo "::error::Docs build job result: ${{ needs.build.result }}"
exit 1
fi
echo "Docs checks passed."

View File

@@ -3,39 +3,58 @@ on:
push:
branches-ignore:
- 'translations**'
paths:
- 'src-ui/**'
- '.github/workflows/ci-frontend.yml'
pull_request:
branches-ignore:
- 'translations**'
paths:
- 'src-ui/**'
- '.github/workflows/ci-frontend.yml'
workflow_dispatch:
concurrency:
group: frontend-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true
jobs:
changes:
name: Detect Frontend Changes
runs-on: ubuntu-24.04
outputs:
frontend_changed: ${{ steps.manual.outputs.frontend || steps.filter.outputs.frontend || 'false' }}
steps:
- name: Checkout
uses: actions/checkout@v6.0.2
with:
fetch-depth: 0
- name: Mark frontend changes for manual runs
id: manual
if: github.event_name == 'workflow_dispatch'
run: echo "frontend=true" >> "$GITHUB_OUTPUT"
- name: Detect frontend changes
id: filter
if: github.event_name != 'workflow_dispatch'
uses: dorny/paths-filter@v3.0.2
with:
filters: |
frontend:
- 'src-ui/**'
- '.github/workflows/ci-frontend.yml'
install-dependencies:
needs: changes
if: needs.changes.outputs.frontend_changed == 'true'
name: Install Dependencies
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
id: cache-frontend-deps
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -45,23 +64,24 @@ jobs:
run: cd src-ui && pnpm install
lint:
name: Lint
needs: install-dependencies
needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -73,7 +93,8 @@ jobs:
run: cd src-ui && pnpm run lint
unit-tests:
name: "Unit Tests (${{ matrix.shard-index }}/${{ matrix.shard-count }})"
needs: install-dependencies
needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-24.04
strategy:
fail-fast: false
@@ -83,19 +104,19 @@ jobs:
shard-count: [4]
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -107,19 +128,20 @@ jobs:
run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }}
- name: Upload test results to Codecov
if: always()
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v5.5.2
with:
flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/
report_type: test_results
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v5
uses: codecov/codecov-action@v5.5.2
with:
flags: frontend-node-${{ matrix.node-version }}
directory: src-ui/coverage/
e2e-tests:
name: "E2E Tests (${{ matrix.shard-index }}/${{ matrix.shard-count }})"
needs: install-dependencies
needs: [changes, install-dependencies]
if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-24.04
container: mcr.microsoft.com/playwright:v1.58.2-noble
env:
@@ -133,19 +155,19 @@ jobs:
shard-count: [2]
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -159,23 +181,24 @@ jobs:
run: cd src-ui && pnpm exec playwright test --shard ${{ matrix.shard-index }}/${{ matrix.shard-count }}
bundle-analysis:
name: Bundle Analysis
needs: [unit-tests, e2e-tests]
needs: [changes, unit-tests, e2e-tests]
if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -187,3 +210,42 @@ jobs:
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: cd src-ui && pnpm run build --configuration=production
gate:
name: Frontend CI Gate
needs: [changes, install-dependencies, lint, unit-tests, e2e-tests, bundle-analysis]
if: always()
runs-on: ubuntu-24.04
steps:
- name: Validate frontend workflow results
run: |
if [[ "${{ needs.changes.outputs.frontend_changed }}" != "true" ]]; then
echo "No frontend-relevant changes detected."
exit 0
fi
if [[ "${{ needs['install-dependencies'].result }}" != "success" ]]; then
echo "::error::Frontend install job result: ${{ needs['install-dependencies'].result }}"
exit 1
fi
if [[ "${{ needs.lint.result }}" != "success" ]]; then
echo "::error::Frontend lint job result: ${{ needs.lint.result }}"
exit 1
fi
if [[ "${{ needs['unit-tests'].result }}" != "success" ]]; then
echo "::error::Frontend unit-tests job result: ${{ needs['unit-tests'].result }}"
exit 1
fi
if [[ "${{ needs['e2e-tests'].result }}" != "success" ]]; then
echo "::error::Frontend e2e-tests job result: ${{ needs['e2e-tests'].result }}"
exit 1
fi
if [[ "${{ needs['bundle-analysis'].result }}" != "success" ]]; then
echo "::error::Frontend bundle-analysis job result: ${{ needs['bundle-analysis'].result }}"
exit 1
fi
echo "Frontend checks passed."

View File

@@ -28,14 +28,14 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
# ---- Frontend Build ----
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
@@ -47,11 +47,11 @@ jobs:
# ---- Backend Setup ----
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6
uses: actions/setup-python@v6.2.0
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@v7.3.1
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -118,7 +118,7 @@ jobs:
sudo chown -R 1000:1000 paperless-ngx/
tar -cJf paperless-ngx.tar.xz paperless-ngx/
- name: Upload release artifact
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v7.0.0
with:
name: release
path: dist/paperless-ngx.tar.xz
@@ -133,7 +133,7 @@ jobs:
version: ${{ steps.get-version.outputs.version }}
steps:
- name: Download release artifact
uses: actions/download-artifact@v7
uses: actions/download-artifact@v8.0.0
with:
name: release
path: ./
@@ -148,7 +148,7 @@ jobs:
fi
- name: Create release and changelog
id: create-release
uses: release-drafter/release-drafter@v6
uses: release-drafter/release-drafter@v6.2.0
with:
name: Paperless-ngx ${{ steps.get-version.outputs.version }}
tag: ${{ steps.get-version.outputs.version }}
@@ -159,7 +159,7 @@ jobs:
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Upload release archive
uses: shogo82148/actions-upload-release-asset@v1
uses: shogo82148/actions-upload-release-asset@v1.9.2
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
upload_url: ${{ steps.create-release.outputs.upload_url }}
@@ -176,16 +176,16 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
with:
ref: main
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6
uses: actions/setup-python@v6.2.0
with:
python-version: ${{ env.DEFAULT_PYTHON_VERSION }}
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@v7.3.1
with:
version: ${{ env.DEFAULT_UV_VERSION }}
enable-cache: true
@@ -218,7 +218,7 @@ jobs:
git commit -am "Changelog ${{ needs.publish-release.outputs.version }} - GHA"
git push origin ${{ needs.publish-release.outputs.version }}-changelog
- name: Create pull request
uses: actions/github-script@v8
uses: actions/github-script@v8.0.0
with:
script: |
const { repo, owner } = context.repo;

View File

@@ -34,10 +34,10 @@ jobs:
# Learn more about CodeQL language support at https://git.io/codeql-language-support
steps:
- name: Checkout repository
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
uses: github/codeql-action/init@v4.32.5
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
@@ -45,4 +45,4 @@ jobs:
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
uses: github/codeql-action/analyze@v4.32.5

View File

@@ -13,11 +13,11 @@ jobs:
runs-on: ubuntu-24.04
steps:
- name: Checkout
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
with:
token: ${{ secrets.PNGX_BOT_PAT }}
- name: crowdin action
uses: crowdin/github-action@v2
uses: crowdin/github-action@v2.15.0
with:
upload_translations: false
download_translations: true

View File

@@ -12,7 +12,7 @@ jobs:
steps:
- name: Label PR by file path or branch name
# see .github/labeler.yml for the labeler config
uses: actions/labeler@v6
uses: actions/labeler@v6.0.1
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}
- name: Label by size
@@ -26,7 +26,7 @@ jobs:
fail_if_xl: 'false'
excluded_files: /\.lock$/ /\.txt$/ ^src-ui/pnpm-lock\.yaml$ ^src-ui/messages\.xlf$ ^src/locale/en_US/LC_MESSAGES/django\.po$
- name: Label by PR title
uses: actions/github-script@v8
uses: actions/github-script@v8.0.0
with:
script: |
const pr = context.payload.pull_request;
@@ -52,7 +52,7 @@ jobs:
}
- name: Label bot-generated PRs
if: ${{ contains(github.actor, 'dependabot') || contains(github.actor, 'crowdin-bot') }}
uses: actions/github-script@v8
uses: actions/github-script@v8.0.0
with:
script: |
const pr = context.payload.pull_request;
@@ -77,7 +77,7 @@ jobs:
}
- name: Welcome comment
if: ${{ !contains(github.actor, 'bot') }}
uses: actions/github-script@v8
uses: actions/github-script@v8.0.0
with:
script: |
const pr = context.payload.pull_request;

View File

@@ -19,6 +19,6 @@ jobs:
if: github.event_name == 'pull_request_target' && (github.event.action == 'opened' || github.event.action == 'reopened') && github.event.pull_request.user.login != 'dependabot'
steps:
- name: Label PR with release-drafter
uses: release-drafter/release-drafter@v6
uses: release-drafter/release-drafter@v6.2.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -15,7 +15,7 @@ jobs:
if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-24.04
steps:
- uses: actions/stale@v10
- uses: actions/stale@v10.2.0
with:
days-before-stale: 7
days-before-close: 14
@@ -37,7 +37,7 @@ jobs:
if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-24.04
steps:
- uses: dessant/lock-threads@v6
- uses: dessant/lock-threads@v6.0.0
with:
issue-inactive-days: '30'
pr-inactive-days: '30'
@@ -57,7 +57,7 @@ jobs:
if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-24.04
steps:
- uses: actions/github-script@v8
- uses: actions/github-script@v8.0.0
with:
script: |
function sleep(ms) {
@@ -114,7 +114,7 @@ jobs:
if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-24.04
steps:
- uses: actions/github-script@v8
- uses: actions/github-script@v8.0.0
with:
script: |
function sleep(ms) {
@@ -206,7 +206,7 @@ jobs:
if: github.repository_owner == 'paperless-ngx'
runs-on: ubuntu-24.04
steps:
- uses: actions/github-script@v8
- uses: actions/github-script@v8.0.0
with:
script: |
function sleep(ms) {

View File

@@ -11,7 +11,7 @@ jobs:
contents: write
steps:
- name: Checkout code
uses: actions/checkout@v6
uses: actions/checkout@v6.0.2
env:
GH_REF: ${{ github.ref }} # sonar rule:githubactions:S7630 - avoid injection
with:
@@ -19,13 +19,13 @@ jobs:
ref: ${{ env.GH_REF }}
- name: Set up Python
id: setup-python
uses: actions/setup-python@v6
uses: actions/setup-python@v6.2.0
- name: Install system dependencies
run: |
sudo apt-get update -qq
sudo apt-get install -qq --no-install-recommends gettext
- name: Install uv
uses: astral-sh/setup-uv@v7
uses: astral-sh/setup-uv@v7.3.1
with:
enable-cache: true
- name: Install backend python dependencies
@@ -36,18 +36,18 @@ jobs:
- name: Generate backend translation strings
run: cd src/ && uv run manage.py makemessages -l en_US -i "samples*"
- name: Install pnpm
uses: pnpm/action-setup@v4
uses: pnpm/action-setup@v4.2.0
with:
version: 10
- name: Use Node.js 24
uses: actions/setup-node@v6
uses: actions/setup-node@v6.2.0
with:
node-version: 24.x
cache: 'pnpm'
cache-dependency-path: 'src-ui/pnpm-lock.yaml'
- name: Cache frontend dependencies
id: cache-frontend-deps
uses: actions/cache@v5
uses: actions/cache@v5.0.3
with:
path: |
~/.pnpm-store
@@ -63,7 +63,7 @@ jobs:
cd src-ui
pnpm run ng extract-i18n
- name: Commit changes
uses: stefanzweifel/git-auto-commit-action@v7
uses: stefanzweifel/git-auto-commit-action@v7.1.0
with:
file_pattern: 'src-ui/messages.xlf src/locale/en_US/LC_MESSAGES/django.po'
commit_message: "Auto translate strings"

View File

@@ -341,6 +341,9 @@ src/documents/migrations/0001_initial.py:0: error: Skipping analyzing "multisele
src/documents/migrations/0001_initial.py:0: error: Skipping analyzing "multiselectfield.db.fields": module is installed, but missing library stubs or py.typed marker [import-untyped]
src/documents/migrations/0008_sharelinkbundle.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/migrations/0008_sharelinkbundle.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/migrations/0012_savedview_visibility_to_ui_settings.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/migrations/0012_savedview_visibility_to_ui_settings.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/migrations/0012_savedview_visibility_to_ui_settings.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/models.py:0: error: Argument 1 to "Path" has incompatible type "Path | None"; expected "str | PathLike[str]" [arg-type]
src/documents/models.py:0: error: Couldn't resolve related manager 'documents' for relation 'documents.models.Document.correspondent'. [django-manager-missing]
src/documents/models.py:0: error: Couldn't resolve related manager 'documents' for relation 'documents.models.Document.document_type'. [django-manager-missing]
@@ -440,9 +443,6 @@ src/documents/permissions.py:0: error: Item "list[str]" of "Any | list[str] | Qu
src/documents/permissions.py:0: error: Item "list[str]" of "Any | list[str] | QuerySet[User, User]" has no attribute "exists" [union-attr]
src/documents/permissions.py:0: error: Missing type parameters for generic type "QuerySet" [type-arg]
src/documents/permissions.py:0: error: Missing type parameters for generic type "dict" [type-arg]
src/documents/plugins/helpers.py:0: error: "Collection[str]" has no attribute "update" [attr-defined]
src/documents/plugins/helpers.py:0: error: Argument 1 to "send" of "BaseStatusManager" has incompatible type "dict[str, Collection[str]]"; expected "dict[str, str | int | None]" [arg-type]
src/documents/plugins/helpers.py:0: error: Argument 1 to "send" of "BaseStatusManager" has incompatible type "dict[str, Collection[str]]"; expected "dict[str, str | int | None]" [arg-type]
src/documents/plugins/helpers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/plugins/helpers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/plugins/helpers.py:0: error: Skipping analyzing "channels_redis.pubsub": module is installed, but missing library stubs or py.typed marker [import-untyped]
@@ -550,6 +550,7 @@ src/documents/serialisers.py:0: error: Function is missing a type annotation [n
src/documents/serialisers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/serialisers.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
@@ -640,6 +641,7 @@ src/documents/serialisers.py:0: error: Missing type parameters for generic type
src/documents/serialisers.py:0: error: Missing type parameters for generic type "Serializer" [type-arg]
src/documents/serialisers.py:0: error: Missing type parameters for generic type "Serializer" [type-arg]
src/documents/serialisers.py:0: error: Missing type parameters for generic type "Serializer" [type-arg]
src/documents/serialisers.py:0: error: Missing type parameters for generic type "Serializer" [type-arg]
src/documents/serialisers.py:0: error: Missing type parameters for generic type "dict" [type-arg]
src/documents/serialisers.py:0: error: Missing type parameters for generic type "dict" [type-arg]
src/documents/serialisers.py:0: error: Need type annotation for "document" [var-annotated]
@@ -667,7 +669,6 @@ src/documents/signals/handlers.py:0: error: Argument 3 to "validate_move" has in
src/documents/signals/handlers.py:0: error: Argument 5 to "_suggestion_printer" has incompatible type "Any | None"; expected "MatchingModel" [arg-type]
src/documents/signals/handlers.py:0: error: Argument 5 to "_suggestion_printer" has incompatible type "Any | None"; expected "MatchingModel" [arg-type]
src/documents/signals/handlers.py:0: error: Argument 5 to "_suggestion_printer" has incompatible type "Any | None"; expected "MatchingModel" [arg-type]
src/documents/signals/handlers.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/signals/handlers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/signals/handlers.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/signals/handlers.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
@@ -1175,6 +1176,14 @@ src/documents/tests/test_management_exporter.py:0: error: Skipping analyzing "al
src/documents/tests/test_management_fuzzy.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/tests/test_management_retagger.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/tests/test_management_superuser.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Cannot determine type of "apps" [has-type]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Cannot determine type of "apps" [has-type]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Incompatible types in assignment (expression has type "str", base class "TestMigrations" defined the type as "None") [assignment]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Incompatible types in assignment (expression has type "str", base class "TestMigrations" defined the type as "None") [assignment]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Incompatible types in assignment (expression has type "str", base class "TestMigrations" defined the type as "None") [assignment]
src/documents/tests/test_migration_saved_view_visibility.py:0: error: Incompatible types in assignment (expression has type "str", base class "TestMigrations" defined the type as "None") [assignment]
src/documents/tests/test_migration_share_link_bundle.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/tests/test_migration_share_link_bundle.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/tests/test_migration_share_link_bundle.py:0: error: Incompatible types in assignment (expression has type "str", base class "TestMigrations" defined the type as "None") [assignment]
@@ -1534,7 +1543,6 @@ src/documents/views.py:0: error: "get_serializer_context" undefined in superclas
src/documents/views.py:0: error: "object" not callable [operator]
src/documents/views.py:0: error: "type[Model]" has no attribute "objects" [attr-defined]
src/documents/views.py:0: error: Argument "path" to "EmailAttachment" has incompatible type "Path | None"; expected "Path" [arg-type]
src/documents/views.py:0: error: Argument 1 to "int" has incompatible type "str | None"; expected "str | Buffer | SupportsInt | SupportsIndex | SupportsTrunc" [arg-type]
src/documents/views.py:0: error: Argument 2 to "match_correspondents" has incompatible type "DocumentClassifier | None"; expected "DocumentClassifier" [arg-type]
src/documents/views.py:0: error: Argument 2 to "match_document_types" has incompatible type "DocumentClassifier | None"; expected "DocumentClassifier" [arg-type]
src/documents/views.py:0: error: Argument 2 to "match_storage_paths" has incompatible type "DocumentClassifier | None"; expected "DocumentClassifier" [arg-type]
@@ -1552,7 +1560,6 @@ src/documents/views.py:0: error: Function is missing a return type annotation [
src/documents/views.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a return type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation [no-untyped-def]
@@ -1609,7 +1616,8 @@ src/documents/views.py:0: error: Function is missing a type annotation [no-unty
src/documents/views.py:0: error: Function is missing a type annotation [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/views.py:0: error: Incompatible type for lookup 'owner': (got "User | AnonymousUser", expected "User | int | None") [misc]
src/documents/views.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/views.py:0: error: Function is missing a type annotation for one or more arguments [no-untyped-def]
src/documents/views.py:0: error: Incompatible types in assignment (expression has type "Any | None", variable has type "dict[Any, Any]") [assignment]
src/documents/views.py:0: error: Incompatible types in assignment (expression has type "QuerySet[Any, Any]", variable has type "list[Any]") [assignment]
src/documents/views.py:0: error: Incompatible types in assignment (expression has type "QuerySet[Any, Any]", variable has type "list[Document]") [assignment]
@@ -1675,11 +1683,11 @@ src/documents/views.py:0: error: Value of type "Iterable[Any]" is not indexable
src/documents/views.py:0: error: Value of type "Iterable[Any]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[Any]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[Any]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[Any]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[CustomField]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[Group]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[MailAccount]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[MailRule]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[SavedView]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[User]" is not indexable [index]
src/documents/views.py:0: error: Value of type "Iterable[Workflow]" is not indexable [index]
src/documents/views.py:0: error: Value of type "dict[str, _PingReply] | None" is not indexable [index]
@@ -1928,6 +1936,7 @@ src/paperless/tests/test_websockets.py:0: error: Item "None" of "BaseChannelLaye
src/paperless/tests/test_websockets.py:0: error: Item "None" of "BaseChannelLayer | None" has no attribute "group_send" [union-attr]
src/paperless/tests/test_websockets.py:0: error: Item "None" of "BaseChannelLayer | None" has no attribute "group_send" [union-attr]
src/paperless/tests/test_websockets.py:0: error: Item "None" of "BaseChannelLayer | None" has no attribute "group_send" [union-attr]
src/paperless/tests/test_websockets.py:0: error: Item "None" of "BaseChannelLayer | None" has no attribute "group_send" [union-attr]
src/paperless/tests/test_websockets.py:0: error: TypedDict "_WebsocketTestScope" has no key "user" [typeddict-item]
src/paperless/tests/test_websockets.py:0: error: TypedDict "_WebsocketTestScope" has no key "user" [typeddict-item]
src/paperless/tests/test_websockets.py:0: error: TypedDict "_WebsocketTestScope" has no key "user" [typeddict-item]

View File

@@ -13,7 +13,9 @@ If you want to implement something big:
## Python
Paperless supports python 3.10 - 3.12 at this time. We format Python code with [ruff](https://docs.astral.sh/ruff/formatter/).
Paperless-ngx currently supports Python 3.11, 3.12, 3.13, and 3.14. As a policy, we aim to support at least the three most recent Python versions, and drop support for versions as they reach end-of-life. Older versions may be supported if dependencies permit, but this is not guaranteed.
We format Python code with [ruff](https://docs.astral.sh/ruff/formatter/).
## Branches

View File

@@ -30,7 +30,7 @@ RUN set -eux \
# Purpose: Installs s6-overlay and rootfs
# Comments:
# - Don't leave anything extra in here either
FROM ghcr.io/astral-sh/uv:0.10.5-python3.12-trixie-slim AS s6-overlay-base
FROM ghcr.io/astral-sh/uv:0.10.7-python3.12-trixie-slim AS s6-overlay-base
WORKDIR /usr/src/s6
@@ -45,7 +45,7 @@ ENV \
ARG TARGETARCH
ARG TARGETVARIANT
# Lock this version
ARG S6_OVERLAY_VERSION=3.2.1.0
ARG S6_OVERLAY_VERSION=3.2.2.0
ARG S6_BUILD_TIME_PKGS="curl \
xz-utils"

View File

@@ -4,7 +4,7 @@
# correct networking for the tests
services:
gotenberg:
image: docker.io/gotenberg/gotenberg:8.26
image: docker.io/gotenberg/gotenberg:8.27
hostname: gotenberg
container_name: gotenberg
network_mode: host

View File

@@ -72,7 +72,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.26
image: docker.io/gotenberg/gotenberg:8.27
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.

View File

@@ -66,7 +66,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.26
image: docker.io/gotenberg/gotenberg:8.27
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.

View File

@@ -55,7 +55,7 @@ services:
PAPERLESS_TIKA_GOTENBERG_ENDPOINT: http://gotenberg:3000
PAPERLESS_TIKA_ENDPOINT: http://tika:9998
gotenberg:
image: docker.io/gotenberg/gotenberg:8.26
image: docker.io/gotenberg/gotenberg:8.27
restart: unless-stopped
# The gotenberg chromium route is used to convert .eml files. We do not
# want to allow external content like tracking pixels or even javascript.

View File

@@ -62,6 +62,10 @@ copies you created in the steps above.
## Updating Paperless {#updating}
!!! warning
Please review the [migration instructions](migration-v3.md) before upgrading Paperless-ngx to v3.0, it includes some breaking changes that require manual intervention before upgrading.
### Docker Route {#docker-updating}
If a new release of paperless-ngx is available, upgrading depends on how

View File

@@ -262,6 +262,10 @@ your files differently, you can do that by adjusting the
or using [storage paths (see below)](#storage-paths). Paperless adds the
correct file extension e.g. `.pdf`, `.jpg` automatically.
When a document has file versions, each version uses the same naming rules and
storage path resolution as any other document file, with an added version suffix
such as `_v1`, `_v2`, etc.
This variable allows you to configure the filename (folders are allowed)
using placeholders. For example, configuring this to
@@ -353,6 +357,8 @@ If paperless detects that two documents share the same filename,
paperless will automatically append `_01`, `_02`, etc to the filename.
This happens if all the placeholders in a filename evaluate to the same
value.
For versioned files, this counter is appended after the version suffix
(for example `statement_v2_01.pdf`).
If there are any errors in the placeholders included in `PAPERLESS_FILENAME_FORMAT`,
paperless will fall back to using the default naming scheme instead.

View File

@@ -466,3 +466,8 @@ Initial API version.
- The document `created` field is now a date, not a datetime. The
`created_date` field is considered deprecated and will be removed in a
future version.
#### Version 10
- The `show_on_dashboard` and `show_in_sidebar` fields of saved views have been
removed. Relevant settings are now stored in the UISettings model.

Binary file not shown.

After

Width:  |  Height:  |  Size: 57 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 61 KiB

View File

@@ -1,9 +1,55 @@
# Changelog
## paperless-ngx 2.20.10
### Bug Fixes
- Fix: support string coercion in filepath jinja templates [@shamoon](https://github.com/shamoon) ([#12244](https://github.com/paperless-ngx/paperless-ngx/pull/12244))
- Fix: apply ordering after annotating tag document count [@shamoon](https://github.com/shamoon) ([#12238](https://github.com/paperless-ngx/paperless-ngx/pull/12238))
- Fix: enforce path limit for db filename fields [@shamoon](https://github.com/shamoon) ([#12235](https://github.com/paperless-ngx/paperless-ngx/pull/12235))
### All App Changes
<details>
<summary>3 changes</summary>
- Fix: support string coercion in filepath jinja templates [@shamoon](https://github.com/shamoon) ([#12244](https://github.com/paperless-ngx/paperless-ngx/pull/12244))
- Fix: apply ordering after annotating tag document count [@shamoon](https://github.com/shamoon) ([#12238](https://github.com/paperless-ngx/paperless-ngx/pull/12238))
- Fix: enforce path limit for db filename fields [@shamoon](https://github.com/shamoon) ([#12235](https://github.com/paperless-ngx/paperless-ngx/pull/12235))
</details>
## paperless-ngx 2.20.9
### Security
- Resolve [GHSA-386h-chg4-cfw9](https://github.com/paperless-ngx/paperless-ngx/security/advisories/GHSA-386h-chg4-cfw9)
### Bug Fixes
- Fixhancement: config option reset [@shamoon](https://github.com/shamoon) ([#12176](https://github.com/paperless-ngx/paperless-ngx/pull/12176))
- Fix: correct page count by separating display vs collection sizes for tags [@shamoon](https://github.com/shamoon) ([#12170](https://github.com/paperless-ngx/paperless-ngx/pull/12170))
### All App Changes
<details>
<summary>2 changes</summary>
- Fixhancement: config option reset [@shamoon](https://github.com/shamoon) ([#12176](https://github.com/paperless-ngx/paperless-ngx/pull/12176))
- Fix: correct page count by separating display vs collection sizes for tags [@shamoon](https://github.com/shamoon) ([#12170](https://github.com/paperless-ngx/paperless-ngx/pull/12170))
</details>
## paperless-ngx 2.20.8
### Security
- Resolve [GHSA-7qqc-wrcw-2fj9](https://github.com/paperless-ngx/paperless-ngx/security/advisories/GHSA-7qqc-wrcw-2fj9)
## paperless-ngx 2.20.7
### Security
- Resolve [GHSA-x395-6h48-wr8v](https://github.com/paperless-ngx/paperless-ngx/security/advisories/GHSA-x395-6h48-wr8v)
### Bug Fixes
- Performance fix: use subqueries to improve object retrieval in large installs [@shamoon](https://github.com/shamoon) ([#11950](https://github.com/paperless-ngx/paperless-ngx/pull/11950))
@@ -22,6 +68,10 @@
## paperless-ngx 2.20.6
### Security
- Resolve [GHSA-jqwv-hx7q-fxh3](https://github.com/paperless-ngx/paperless-ngx/security/advisories/GHSA-jqwv-hx7q-fxh3) and [GHSA-w47q-3m69-84v8](https://github.com/paperless-ngx/paperless-ngx/security/advisories/GHSA-w47q-3m69-84v8)
### Bug Fixes
- Fix: extract all ids for nested tags [@shamoon](https://github.com/shamoon) ([#11888](https://github.com/paperless-ngx/paperless-ngx/pull/11888))

View File

@@ -51,137 +51,172 @@ matcher.
### Database
By default, Paperless uses **SQLite** with a database stored at `data/db.sqlite3`.
To switch to **PostgreSQL** or **MariaDB**, set [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) and optionally configure other
database-related environment variables.
For multi-user or higher-throughput deployments, **PostgreSQL** (recommended) or
**MariaDB** can be used instead by setting [`PAPERLESS_DBENGINE`](#PAPERLESS_DBENGINE)
and the relevant connection variables.
#### [`PAPERLESS_DBENGINE=<engine>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
: Specifies the database engine to use. Accepted values are `sqlite`, `postgresql`,
and `mariadb`.
Defaults to `sqlite` if not set.
PostgreSQL and MariaDB both require [`PAPERLESS_DBHOST`](#PAPERLESS_DBHOST) to be
set. SQLite does not use any other connection variables; the database file is always
located at `<PAPERLESS_DATA_DIR>/db.sqlite3`.
!!! warning
Using MariaDB comes with some caveats.
See [MySQL Caveats](advanced_usage.md#mysql-caveats).
#### [`PAPERLESS_DBHOST=<hostname>`](#PAPERLESS_DBHOST) {#PAPERLESS_DBHOST}
: If unset, Paperless uses **SQLite** by default.
Set `PAPERLESS_DBHOST` to switch to PostgreSQL or MariaDB instead.
#### [`PAPERLESS_DBENGINE=<engine_name>`](#PAPERLESS_DBENGINE) {#PAPERLESS_DBENGINE}
: Optional. Specifies the database engine to use when connecting to a remote database.
Available options are `postgresql` and `mariadb`.
Defaults to `postgresql` if `PAPERLESS_DBHOST` is set.
!!! warning
Using MariaDB comes with some caveats. See [MySQL Caveats](advanced_usage.md#mysql-caveats).
: Hostname of the PostgreSQL or MariaDB database server. Required when
`PAPERLESS_DBENGINE` is `postgresql` or `mariadb`.
#### [`PAPERLESS_DBPORT=<port>`](#PAPERLESS_DBPORT) {#PAPERLESS_DBPORT}
: Port to use when connecting to PostgreSQL or MariaDB.
Default is `5432` for PostgreSQL and `3306` for MariaDB.
Defaults to `5432` for PostgreSQL and `3306` for MariaDB.
#### [`PAPERLESS_DBNAME=<name>`](#PAPERLESS_DBNAME) {#PAPERLESS_DBNAME}
: Name of the database to connect to when using PostgreSQL or MariaDB.
: Name of the PostgreSQL or MariaDB database to connect to.
Defaults to "paperless".
Defaults to `paperless`.
#### [`PAPERLESS_DBUSER=<name>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER}
#### [`PAPERLESS_DBUSER=<user>`](#PAPERLESS_DBUSER) {#PAPERLESS_DBUSER}
: Username for authenticating with the PostgreSQL or MariaDB database.
Defaults to "paperless".
Defaults to `paperless`.
#### [`PAPERLESS_DBPASS=<password>`](#PAPERLESS_DBPASS) {#PAPERLESS_DBPASS}
: Password for the PostgreSQL or MariaDB database user.
Defaults to "paperless".
Defaults to `paperless`.
#### [`PAPERLESS_DBSSLMODE=<mode>`](#PAPERLESS_DBSSLMODE) {#PAPERLESS_DBSSLMODE}
#### [`PAPERLESS_DB_OPTIONS=<options>`](#PAPERLESS_DB_OPTIONS) {#PAPERLESS_DB_OPTIONS}
: SSL mode to use when connecting to PostgreSQL or MariaDB.
: Advanced database connection options as a semicolon-delimited key-value string.
Keys and values are separated by `=`. Dot-notation produces nested option
dictionaries; for example, `pool.max_size=20` sets
`OPTIONS["pool"]["max_size"] = 20`.
See [the official documentation about
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
Options specified here are merged over the engine defaults. Unrecognised keys
are passed through to the underlying database driver without validation, so a
typo will be silently ignored rather than producing an error.
See [the official documentation about
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-mode).
Refer to your database driver's documentation for the full set of accepted keys:
*Note*: SSL mode values differ between PostgreSQL and MariaDB.
- PostgreSQL: [libpq connection parameters](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-PARAMKEYWORDS)
- MariaDB: [MariaDB Connector/Python](https://mariadb.com/kb/en/mariadb-connector-python/)
- SQLite: [SQLite PRAGMA statements](https://www.sqlite.org/pragma.html)
Default is `prefer` for PostgreSQL and `PREFERRED` for MariaDB.
!!! note "PostgreSQL connection pooling"
#### [`PAPERLESS_DBSSLROOTCERT=<ca-path>`](#PAPERLESS_DBSSLROOTCERT) {#PAPERLESS_DBSSLROOTCERT}
Pool size is controlled via `pool.min_size` and `pool.max_size`. When
configuring pooling, ensure your PostgreSQL `max_connections` is large enough
to handle all pool connections across all workers:
`(web_workers + celery_workers) * pool.max_size + safety_margin`.
: Path to the SSL root certificate used to verify the database server.
**Examples:**
See [the official documentation about
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
Changes the location of `root.crt`.
```bash title="PostgreSQL: require SSL, set a custom CA certificate, and limit the pool size"
PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=5"
```
See [the official documentation about
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-ca).
```bash title="MariaDB: require SSL with a custom CA certificate"
PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED;ssl.ca=/certs/ca.pem"
```
Defaults to unset, using the standard location in the home directory.
```bash title="SQLite: set a busy timeout of 30 seconds"
# PostgreSQL: set a connection timeout
PAPERLESS_DB_OPTIONS="connect_timeout=10"
```
#### [`PAPERLESS_DBSSLCERT=<client-cert-path>`](#PAPERLESS_DBSSLCERT) {#PAPERLESS_DBSSLCERT}
#### ~~[`PAPERLESS_DBSSLMODE`](#PAPERLESS_DBSSLMODE)~~ {#PAPERLESS_DBSSLMODE}
: Path to the client SSL certificate used when connecting securely.
!!! failure "Removed in v3"
See [the official documentation about
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
See [the official documentation about
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-cert).
```bash title="PostgreSQL"
PAPERLESS_DB_OPTIONS="sslmode=require"
```
Changes the location of `postgresql.crt`.
```bash title="MariaDB"
PAPERLESS_DB_OPTIONS="ssl_mode=REQUIRED"
```
Defaults to unset, using the standard location in the home directory.
#### ~~[`PAPERLESS_DBSSLROOTCERT`](#PAPERLESS_DBSSLROOTCERT)~~ {#PAPERLESS_DBSSLROOTCERT}
#### [`PAPERLESS_DBSSLKEY=<client-cert-key>`](#PAPERLESS_DBSSLKEY) {#PAPERLESS_DBSSLKEY}
!!! failure "Removed in v3"
: Path to the client SSL private key used when connecting securely.
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
See [the official documentation about
sslmode for PostgreSQL](https://www.postgresql.org/docs/current/libpq-ssl.html).
```bash title="PostgreSQL"
PAPERLESS_DB_OPTIONS="sslrootcert=/path/to/ca.pem"
```
See [the official documentation about
sslmode for MySQL and MariaDB](https://dev.mysql.com/doc/refman/8.0/en/connection-options.html#option_general_ssl-key).
```bash title="MariaDB"
PAPERLESS_DB_OPTIONS="ssl.ca=/path/to/ca.pem"
```
Changes the location of `postgresql.key`.
#### ~~[`PAPERLESS_DBSSLCERT`](#PAPERLESS_DBSSLCERT)~~ {#PAPERLESS_DBSSLCERT}
Defaults to unset, using the standard location in the home directory.
!!! failure "Removed in v3"
#### [`PAPERLESS_DB_TIMEOUT=<int>`](#PAPERLESS_DB_TIMEOUT) {#PAPERLESS_DB_TIMEOUT}
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
: Sets how long a database connection should wait before timing out.
```bash title="PostgreSQL"
PAPERLESS_DB_OPTIONS="sslcert=/path/to/client.crt"
```
For SQLite, this sets how long to wait if the database is locked.
For PostgreSQL or MariaDB, this sets the connection timeout.
```bash title="MariaDB"
PAPERLESS_DB_OPTIONS="ssl.cert=/path/to/client.crt"
```
Defaults to unset, which uses Djangos built-in defaults.
#### ~~[`PAPERLESS_DBSSLKEY`](#PAPERLESS_DBSSLKEY)~~ {#PAPERLESS_DBSSLKEY}
#### [`PAPERLESS_DB_POOLSIZE=<int>`](#PAPERLESS_DB_POOLSIZE) {#PAPERLESS_DB_POOLSIZE}
!!! failure "Removed in v3"
: Defines the maximum number of database connections to keep in the pool.
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
Only applies to PostgreSQL. This setting is ignored for other database engines.
```bash title="PostgreSQL"
PAPERLESS_DB_OPTIONS="sslkey=/path/to/client.key"
```
The value must be greater than or equal to 1 to be used.
Defaults to unset, which disables connection pooling.
```bash title="MariaDB"
PAPERLESS_DB_OPTIONS="ssl.key=/path/to/client.key"
```
!!! note
#### ~~[`PAPERLESS_DB_TIMEOUT`](#PAPERLESS_DB_TIMEOUT)~~ {#PAPERLESS_DB_TIMEOUT}
A pool of 8-10 connections per worker is typically sufficient.
If you encounter error messages such as `couldn't get a connection`
or database connection timeouts, you probably need to increase the pool size.
!!! failure "Removed in v3"
!!! warning
Make sure your PostgreSQL `max_connections` setting is large enough to handle the connection pools:
`(NB_PAPERLESS_WORKERS + NB_CELERY_WORKERS) × POOL_SIZE + SAFETY_MARGIN`. For example, with
4 Paperless workers and 2 Celery workers, and a pool size of 8:``(4 + 2) × 8 + 10 = 58`,
so `max_connections = 60` (or even more) is appropriate.
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
This assumes only Paperless-ngx connects to your PostgreSQL instance. If you have other applications,
you should increase `max_connections` accordingly.
```bash title="SQLite"
PAPERLESS_DB_OPTIONS="timeout=30"
```
```bash title="PostgreSQL or MariaDB"
PAPERLESS_DB_OPTIONS="connect_timeout=30"
```
#### ~~[`PAPERLESS_DB_POOLSIZE`](#PAPERLESS_DB_POOLSIZE)~~ {#PAPERLESS_DB_POOLSIZE}
!!! failure "Removed in v3"
Use [`PAPERLESS_DB_OPTIONS`](#PAPERLESS_DB_OPTIONS) instead.
```bash
PAPERLESS_DB_OPTIONS="pool.max_size=10"
```
#### [`PAPERLESS_DB_READ_CACHE_ENABLED=<bool>`](#PAPERLESS_DB_READ_CACHE_ENABLED) {#PAPERLESS_DB_READ_CACHE_ENABLED}

View File

@@ -48,3 +48,58 @@ The `CONSUMER_BARCODE_SCANNER` setting has been removed. zxing-cpp is now the on
reliability.
- The `libzbar0` / `libzbar-dev` system packages are no longer required and can be removed from any custom Docker
images or host installations.
## Database Engine
`PAPERLESS_DBENGINE` is now required to use PostgreSQL or MariaDB. Previously, the
engine was inferred from the presence of `PAPERLESS_DBHOST`, with `PAPERLESS_DBENGINE`
only needed to select MariaDB over PostgreSQL.
SQLite users require no changes, though they may explicitly set their engine if desired.
#### Action Required
PostgreSQL and MariaDB users must add `PAPERLESS_DBENGINE` to their environment:
```yaml
# v2 (PostgreSQL inferred from PAPERLESS_DBHOST)
PAPERLESS_DBHOST: postgres
# v3 (engine must be explicit)
PAPERLESS_DBENGINE: postgresql
PAPERLESS_DBHOST: postgres
```
See [`PAPERLESS_DBENGINE`](configuration.md#PAPERLESS_DBENGINE) for accepted values.
## Database Advanced Options
The individual SSL, timeout, and pooling variables have been removed in favor of a
single [`PAPERLESS_DB_OPTIONS`](configuration.md#PAPERLESS_DB_OPTIONS) string. This
consolidates a growing set of engine-specific variables into one place, and allows
any option supported by the underlying database driver to be set without requiring a
dedicated environment variable for each.
The removed variables and their replacements are:
| Removed Variable | Replacement in `PAPERLESS_DB_OPTIONS` |
| ------------------------- | ---------------------------------------------------------------------------- |
| `PAPERLESS_DBSSLMODE` | `sslmode=<value>` (PostgreSQL) or `ssl_mode=<value>` (MariaDB) |
| `PAPERLESS_DBSSLROOTCERT` | `sslrootcert=<path>` (PostgreSQL) or `ssl.ca=<path>` (MariaDB) |
| `PAPERLESS_DBSSLCERT` | `sslcert=<path>` (PostgreSQL) or `ssl.cert=<path>` (MariaDB) |
| `PAPERLESS_DBSSLKEY` | `sslkey=<path>` (PostgreSQL) or `ssl.key=<path>` (MariaDB) |
| `PAPERLESS_DB_POOLSIZE` | `pool.max_size=<value>` (PostgreSQL only) |
| `PAPERLESS_DB_TIMEOUT` | `timeout=<value>` (SQLite) or `connect_timeout=<value>` (PostgreSQL/MariaDB) |
The deprecated variables will continue to function for now but will be removed in a
future release. A deprecation warning is logged at startup for each deprecated variable
that is still set.
#### Action Required
Users with any of the deprecated variables set should migrate to `PAPERLESS_DB_OPTIONS`.
Multiple options are combined in a single value:
```bash
PAPERLESS_DB_OPTIONS="sslmode=require;sslrootcert=/certs/ca.pem;pool.max_size=10"
```

View File

@@ -172,7 +172,7 @@ to enable polling and disable inotify. See [here](configuration.md#polling).
#### Prerequisites
- Paperless runs on Linux only, Windows is not supported.
- Python 3 is required with versions 3.10 - 3.12 currently supported. Newer versions may work, but some dependencies may not be fully compatible.
- Python 3.11, 3.12, 3.13, or 3.14 is required. As a policy, Paperless-ngx aims to support at least the three most recent Python versions and drops support for versions as they reach end-of-life. Newer versions may work, but some dependencies may not be fully compatible.
#### Installation
@@ -504,9 +504,8 @@ installation. Keep these points in mind:
- Read the [changelog](changelog.md) and
take note of breaking changes.
- Decide whether to stay on SQLite or migrate to PostgreSQL.
See [documentation](#sqlite_to_psql) for details on moving data
from SQLite to PostgreSQL. Both work fine with
Paperless. However, if you already have a database server running
Both work fine with Paperless-ngx.
However, if you already have a database server running
for other services, you might as well use it for Paperless as well.
- The task scheduler of Paperless, which is used to execute periodic
tasks such as email checking and maintenance, requires a

View File

@@ -95,6 +95,7 @@ Think of versions as **file history** for a document.
- Versions track the underlying file and extracted text content (OCR/text).
- Metadata such as tags, correspondent, document type, storage path and custom fields stay on the "root" document.
- Version files follow normal filename formatting (including storage paths) and add a `_vN` suffix (for example `_v1`, `_v2`).
- By default, search and document content use the latest version.
- In document detail, selecting a version switches the preview, file metadata and content (and download etc buttons) to that version.
- Deleting a non-root version keeps metadata and falls back to the latest remaining version.
@@ -616,7 +617,7 @@ applied. You can use the following placeholders in the template with any trigger
- `{{added_day}}`: added day
- `{{added_time}}`: added time in HH:MM format
- `{{original_filename}}`: original file name without extension
- `{{filename}}`: current file name without extension
- `{{filename}}`: current file name without extension (for "added" workflows this may not be final yet, you can use `{{original_filename}}`)
- `{{doc_title}}`: current document title (cannot be used in title assignment)
The following placeholders are only available for "added" or "updated" triggers
@@ -626,7 +627,7 @@ The following placeholders are only available for "added" or "updated" triggers
- `{{created_year_short}}`: created year
- `{{created_month}}`: created month
- `{{created_month_name}}`: created month name
- `{created_month_name_short}}`: created month short name
- `{{created_month_name_short}}`: created month short name
- `{{created_day}}`: created day
- `{{created_time}}`: created time in HH:MM format
- `{{doc_url}}`: URL to the document in the web UI. Requires the `PAPERLESS_URL` setting to be set.

View File

@@ -1,12 +1,11 @@
[project]
name = "paperless-ngx"
version = "2.20.8"
version = "2.20.10"
description = "A community-supported supercharged document management system: scan, index and archive all your physical documents"
readme = "README.md"
requires-python = ">=3.10"
requires-python = ">=3.11"
classifiers = [
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
@@ -37,6 +36,7 @@ dependencies = [
"django-filter~=25.1",
"django-guardian~=3.3.0",
"django-multiselectfield~=1.0.1",
"django-rich~=2.2.0",
"django-soft-delete~=1.0.18",
"django-treenode>=0.23.2",
"djangorestframework~=3.16",
@@ -76,7 +76,6 @@ dependencies = [
"setproctitle~=1.3.4",
"tika-client~=0.10.0",
"torch~=2.10.0",
"tqdm~=4.67.1",
"watchfiles>=1.1.1",
"whitenoise~=6.11",
"whoosh-reloaded>=2.7.5",
@@ -111,6 +110,7 @@ docs = [
testing = [
"daphne",
"factory-boy~=3.3.1",
"faker~=40.5.1",
"imagehash",
"pytest~=9.0.0",
"pytest-cov~=7.0.0",
@@ -149,7 +149,6 @@ typing = [
"types-pytz",
"types-redis",
"types-setuptools",
"types-tqdm",
]
[tool.uv]
@@ -177,7 +176,7 @@ torch = [
]
[tool.ruff]
target-version = "py310"
target-version = "py311"
line-length = 88
src = [
"src",
@@ -304,11 +303,13 @@ markers = [
"tika: Tests requiring Tika service",
"greenmail: Tests requiring Greenmail service",
"date_parsing: Tests which cover date parsing from content or filename",
"management: Tests which cover management commands/functionality",
]
[tool.pytest_env]
PAPERLESS_DISABLE_DBHANDLER = "true"
PAPERLESS_CACHE_BACKEND = "django.core.cache.backends.locmem.LocMemCache"
PAPERLESS_CHANNELS_BACKEND = "channels.layers.InMemoryChannelLayer"
[tool.coverage.report]
exclude_also = [

View File

@@ -58,7 +58,7 @@
"content": {
"size": -1,
"mimeType": "application/json",
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true,\"dashboard_views_visible_ids\":[7,4],\"sidebar_views_visible_ids\":[7,4,11]},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
},
"headersSize": -1,
"bodySize": -1,

View File

@@ -58,7 +58,7 @@
"content": {
"size": -1,
"mimeType": "application/json",
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true,\"dashboard_views_visible_ids\":[7,4],\"sidebar_views_visible_ids\":[7,4,11]},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
},
"headersSize": -1,
"bodySize": -1,

View File

@@ -58,7 +58,7 @@
"content": {
"size": -1,
"mimeType": "application/json",
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true,\"dashboard_views_visible_ids\":[7,4],\"sidebar_views_visible_ids\":[7,4,11]},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
},
"headersSize": -1,
"bodySize": -1,

View File

@@ -58,7 +58,7 @@
"content": {
"size": -1,
"mimeType": "application/json",
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
"text": "{\"user\":{\"id\":2,\"username\":\"testuser\",\"is_superuser\":false,\"groups\":[]},\"settings\":{\"language\":\"\",\"bulk_edit\":{\"confirmation_dialogs\":true,\"apply_on_close\":false},\"documentListSize\":50,\"dark_mode\":{\"use_system\":false,\"enabled\":\"false\",\"thumb_inverted\":\"true\"},\"theme\":{\"color\":\"#9fbf2f\"},\"document_details\":{\"native_pdf_viewer\":false},\"date_display\":{\"date_locale\":\"\",\"date_format\":\"mediumDate\"},\"notifications\":{\"consumer_new_documents\":true,\"consumer_success\":true,\"consumer_failed\":true,\"consumer_suppress_on_dashboard\":true},\"comments_enabled\":true,\"slim_sidebar\":false,\"update_checking\":{\"enabled\":false,\"backend_setting\":\"default\"},\"saved_views\":{\"warn_on_unsaved_change\":true,\"dashboard_views_visible_ids\":[7,4],\"sidebar_views_visible_ids\":[7,4,11]},\"notes_enabled\":true,\"tour_complete\":true},\"permissions\":[\"delete_schedule\",\"view_note\",\"view_taskattributes\",\"delete_ormq\",\"add_ormq\",\"change_tag\",\"change_chordcounter\",\"delete_savedviewfilterrule\",\"delete_comment\",\"add_session\",\"add_mailrule\",\"add_tokenproxy\",\"delete_taskresult\",\"view_failure\",\"add_tag\",\"view_savedviewfilterrule\",\"view_paperlesstask\",\"change_mailaccount\",\"change_frontendsettings\",\"delete_group\",\"add_userobjectpermission\",\"add_failure\",\"delete_mailrule\",\"view_userobjectpermission\",\"change_schedule\",\"delete_note\",\"view_ormq\",\"add_note\",\"add_chordcounter\",\"delete_token\",\"change_failure\",\"add_savedviewfilterrule\",\"delete_user\",\"view_correspondent\",\"view_schedule\",\"change_tokenproxy\",\"view_user\",\"delete_task\",\"delete_correspondent\",\"delete_chordcounter\",\"delete_document\",\"view_taskresult\",\"change_document\",\"add_frontendsettings\",\"view_mailrule\",\"change_ormq\",\"delete_taskattributes\",\"view_logentry\",\"view_mailaccount\",\"view_log\",\"delete_success\",\"view_frontendsettings\",\"view_documenttype\",\"change_taskresult\",\"view_permission\",\"add_groupobjectpermission\",\"change_user\",\"view_document\",\"change_userobjectpermission\",\"add_user\",\"add_correspondent\",\"add_token\",\"add_mailaccount\",\"change_group\",\"add_group\",\"delete_processedmail\",\"delete_contenttype\",\"add_savedview\",\"view_chordcounter\",\"delete_tokenproxy\",\"change_groupresult\",\"delete_session\",\"view_savedview\",\"view_processedmail\",\"add_comment\",\"view_storagepath\",\"delete_documenttype\",\"add_processedmail\",\"view_group\",\"change_processedmail\",\"view_session\",\"delete_storagepath\",\"delete_paperlesstask\",\"add_groupresult\",\"delete_savedview\",\"delete_userobjectpermission\",\"view_tokenproxy\",\"add_task\",\"view_tag\",\"add_taskresult\",\"change_documenttype\",\"change_mailrule\",\"add_document\",\"change_comment\",\"view_task\",\"view_groupresult\",\"change_contenttype\",\"view_groupobjectpermission\",\"change_task\",\"add_log\",\"add_success\",\"change_savedview\",\"delete_frontendsettings\",\"view_success\",\"add_permission\",\"change_correspondent\",\"add_paperlesstask\",\"change_paperlesstask\",\"add_contenttype\",\"view_comment\",\"change_logentry\",\"delete_logentry\",\"delete_mailaccount\",\"change_session\",\"delete_groupresult\",\"add_logentry\",\"change_savedviewfilterrule\",\"change_success\",\"delete_tag\",\"add_taskattributes\",\"change_groupobjectpermission\",\"delete_failure\",\"add_uisettings\",\"view_token\",\"add_schedule\",\"delete_log\",\"delete_uisettings\",\"change_permission\",\"delete_groupobjectpermission\",\"change_token\",\"view_uisettings\",\"change_uisettings\",\"delete_permission\",\"add_storagepath\",\"change_storagepath\",\"view_contenttype\",\"change_note\",\"change_log\",\"change_taskattributes\",\"add_documenttype\"]}"
},
"headersSize": -1,
"bodySize": -1,

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
{
"name": "paperless-ngx-ui",
"version": "2.20.8",
"version": "2.20.10",
"scripts": {
"preinstall": "npx only-allow pnpm",
"ng": "ng",
@@ -11,17 +11,17 @@
},
"private": true,
"dependencies": {
"@angular/cdk": "^21.1.3",
"@angular/common": "~21.1.3",
"@angular/compiler": "~21.1.3",
"@angular/core": "~21.1.3",
"@angular/forms": "~21.1.3",
"@angular/localize": "~21.1.3",
"@angular/platform-browser": "~21.1.3",
"@angular/platform-browser-dynamic": "~21.1.3",
"@angular/router": "~21.1.3",
"@angular/cdk": "^21.2.0",
"@angular/common": "~21.2.0",
"@angular/compiler": "~21.2.0",
"@angular/core": "~21.2.0",
"@angular/forms": "~21.2.0",
"@angular/localize": "~21.2.0",
"@angular/platform-browser": "~21.2.0",
"@angular/platform-browser-dynamic": "~21.2.0",
"@angular/router": "~21.2.0",
"@ng-bootstrap/ng-bootstrap": "^20.0.0",
"@ng-select/ng-select": "^21.2.0",
"@ng-select/ng-select": "^21.4.1",
"@ngneat/dirty-check-forms": "^3.0.3",
"@popperjs/core": "^2.11.8",
"bootstrap": "^5.3.8",
@@ -37,25 +37,25 @@
"tslib": "^2.8.1",
"utif": "^3.1.0",
"uuid": "^13.0.0",
"zone.js": "^0.16.0"
"zone.js": "^0.16.1"
},
"devDependencies": {
"@angular-builders/custom-webpack": "^21.0.3",
"@angular-builders/jest": "^21.0.3",
"@angular-devkit/core": "^21.1.3",
"@angular-devkit/schematics": "^21.1.3",
"@angular-devkit/core": "^21.2.0",
"@angular-devkit/schematics": "^21.2.0",
"@angular-eslint/builder": "21.2.0",
"@angular-eslint/eslint-plugin": "21.2.0",
"@angular-eslint/eslint-plugin-template": "21.2.0",
"@angular-eslint/schematics": "21.2.0",
"@angular-eslint/template-parser": "21.2.0",
"@angular/build": "^21.1.3",
"@angular/cli": "~21.1.3",
"@angular/compiler-cli": "~21.1.3",
"@angular/build": "^21.2.0",
"@angular/cli": "~21.2.0",
"@angular/compiler-cli": "~21.2.0",
"@codecov/webpack-plugin": "^1.9.1",
"@playwright/test": "^1.58.2",
"@types/jest": "^30.0.0",
"@types/node": "^25.2.1",
"@types/node": "^25.3.3",
"@typescript-eslint/eslint-plugin": "^8.54.0",
"@typescript-eslint/parser": "^8.54.0",
"@typescript-eslint/utils": "^8.54.0",
@@ -63,12 +63,12 @@
"jest": "30.2.0",
"jest-environment-jsdom": "^30.2.0",
"jest-junit": "^16.0.0",
"jest-preset-angular": "^16.0.0",
"jest-preset-angular": "^16.1.1",
"jest-websocket-mock": "^2.5.0",
"prettier-plugin-organize-imports": "^4.3.0",
"ts-node": "~10.9.1",
"typescript": "^5.9.3",
"webpack": "^5.105.0"
"webpack": "^5.105.3"
},
"packageManager": "pnpm@10.17.1",
"pnpm": {

2597
src-ui/pnpm-lock.yaml generated

File diff suppressed because it is too large Load Diff

View File

@@ -19,13 +19,18 @@
<div class="col">
<div class="card bg-light">
<div class="card-body">
<div class="card-title">
<h6>
{{option.title}}
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
<div class="card-title d-flex align-items-center">
<h6 class="mb-0">
{{option.title}}
</h6>
<a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
<i-bs name="info-circle"></i-bs>
</a>
@if (isSet(option.key)) {
<button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
<i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
</button>
}
</div>
<div class="mb-n3">
@switch (option.type) {

View File

@@ -144,4 +144,18 @@ describe('ConfigComponent', () => {
component.uploadFile(new File([], 'test.png'), 'app_logo')
expect(initSpy).toHaveBeenCalled()
})
it('should reset option to null', () => {
component.configForm.patchValue({ output_type: OutputTypeConfig.PDF_A })
expect(component.isSet('output_type')).toBeTruthy()
component.resetOption('output_type')
expect(component.configForm.get('output_type').value).toBeNull()
expect(component.isSet('output_type')).toBeFalsy()
component.configForm.patchValue({ app_title: 'Test Title' })
component.resetOption('app_title')
expect(component.configForm.get('app_title').value).toBeNull()
component.configForm.patchValue({ barcodes_enabled: true })
component.resetOption('barcodes_enabled')
expect(component.configForm.get('barcodes_enabled').value).toBeNull()
})
})

View File

@@ -210,4 +210,12 @@ export class ConfigComponent
},
})
}
public isSet(key: string): boolean {
return this.configForm.get(key).value != null
}
public resetOption(key: string) {
this.configForm.get(key).setValue(null)
}
}

View File

@@ -277,7 +277,7 @@
<div class="col">
<select class="form-select" formControlName="pdfEditorDefaultEditMode">
<option [ngValue]="PdfEditorEditMode.Create" i18n>Create new document(s)</option>
<option [ngValue]="PdfEditorEditMode.Update" i18n>Update existing document</option>
<option [ngValue]="PdfEditorEditMode.Update" i18n>Add document version</option>
</select>
</div>
</div>

View File

@@ -99,12 +99,7 @@
</ul>
<div class="nav-group mt-3 mb-1" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
@if (savedViewService.loading) {
<h6 class="sidebar-heading px-3 text-muted">
<span i18n>Saved views</span>
<div class="spinner-border spinner-border-sm fw-normal ms-2" role="status"></div>
</h6>
} @else if (savedViewService.sidebarViews?.length > 0) {
@if (savedViewService.sidebarViews?.length > 0) {
<h6 class="sidebar-heading px-3 text-muted">
<span i18n>Saved views</span>
</h6>
@@ -134,6 +129,11 @@
</li>
}
</ul>
} @else if (savedViewService.loading) {
<h6 class="sidebar-heading px-3 text-muted">
<span i18n>Saved views</span>
<div class="spinner-border spinner-border-sm fw-normal ms-2" role="status"></div>
</h6>
}
</div>

View File

@@ -84,7 +84,7 @@
<input type="radio" class="btn-check" [(ngModel)]="editMode" [value]="PdfEditorEditMode.Update" id="editModeUpdate" name="editmode" [disabled]="hasSplit()">
<label for="editModeUpdate" class="btn btn-outline-primary btn-sm">
<i-bs name="pencil"></i-bs>
<span class="form-check-label ms-2" i18n>Update existing document</span>
<span class="form-check-label ms-2" i18n>Add document version</span>
</label>
</div>
@if (editMode === PdfEditorEditMode.Create) {

View File

@@ -16,6 +16,12 @@
</div>
</form>
@if (note) {
<div class="small text-muted fst-italic mt-2">
{{ note }}
</div>
}
</div>
<div class="modal-footer">
@if (!buttonsEnabled) {

View File

@@ -40,6 +40,9 @@ export class PermissionsDialogComponent {
@Input()
title = $localize`Set permissions`
@Input()
note: string = null
@Input()
set object(o: ObjectWithPermissions) {
this.o = o

View File

@@ -65,6 +65,7 @@ import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { WebsocketStatusService } from 'src/app/services/websocket-status.service'
import { environment } from 'src/environments/environment'
import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
@@ -83,9 +84,9 @@ const doc: Document = {
storage_path: 31,
tags: [41, 42, 43],
content: 'text content',
added: new Date('May 4, 2014 03:24:00'),
created: new Date('May 4, 2014 03:24:00'),
modified: new Date('May 4, 2014 03:24:00'),
added: new Date('May 4, 2014 03:24:00').toISOString(),
created: new Date('May 4, 2014 03:24:00').toISOString(),
modified: new Date('May 4, 2014 03:24:00').toISOString(),
archive_serial_number: null,
original_file_name: 'file.pdf',
owner: null,
@@ -327,6 +328,29 @@ describe('DocumentDetailComponent', () => {
expect(component.activeNavID).toEqual(component.DocumentDetailNavIDs.Notes)
})
it('should switch from preview to details when pdf preview enters the DOM', fakeAsync(() => {
component.nav = {
activeId: component.DocumentDetailNavIDs.Preview,
select: jest.fn(),
} as any
;(component as any).pdfPreview = {
nativeElement: { offsetParent: {} },
}
tick()
expect(component.nav.select).toHaveBeenCalledWith(
component.DocumentDetailNavIDs.Details
)
}))
it('should forward title key up value to titleSubject', () => {
const subjectSpy = jest.spyOn(component.titleSubject, 'next')
component.titleKeyUp({ target: { value: 'Updated title' } })
expect(subjectSpy).toHaveBeenCalledWith('Updated title')
})
it('should change url on tab switch', () => {
initNormally()
const navigateSpy = jest.spyOn(router, 'navigate')
@@ -524,7 +548,7 @@ describe('DocumentDetailComponent', () => {
jest.spyOn(documentService, 'get').mockReturnValue(
of({
...doc,
modified: new Date('2024-01-02T00:00:00Z'),
modified: '2024-01-02T00:00:00Z',
duplicate_documents: updatedDuplicates,
})
)
@@ -1386,17 +1410,21 @@ describe('DocumentDetailComponent', () => {
expect(errorSpy).toHaveBeenCalled()
})
it('should warn when open document does not match doc retrieved from backend on init', () => {
it('should show incoming update modal when open local draft is older than backend on init', () => {
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modals) => (openModal = modals[0]))
const modalSpy = jest.spyOn(modalService, 'open')
const openDoc = Object.assign({}, doc)
const openDoc = Object.assign({}, doc, {
__changedFields: ['title'],
})
// simulate a document being modified elsewhere and db updated
doc.modified = new Date()
const remoteDoc = Object.assign({}, doc, {
modified: new Date(new Date(doc.modified).getTime() + 1000).toISOString(),
})
jest
.spyOn(activatedRoute, 'paramMap', 'get')
.mockReturnValue(of(convertToParamMap({ id: 3, section: 'details' })))
jest.spyOn(documentService, 'get').mockReturnValueOnce(of(doc))
jest.spyOn(documentService, 'get').mockReturnValueOnce(of(remoteDoc))
jest.spyOn(openDocumentsService, 'getOpenDocument').mockReturnValue(openDoc)
jest.spyOn(customFieldsService, 'listAll').mockReturnValue(
of({
@@ -1406,11 +1434,185 @@ describe('DocumentDetailComponent', () => {
})
)
fixture.detectChanges() // calls ngOnInit
expect(modalSpy).toHaveBeenCalledWith(ConfirmDialogComponent)
const closeSpy = jest.spyOn(openModal, 'close')
expect(modalSpy).toHaveBeenCalledWith(ConfirmDialogComponent, {
backdrop: 'static',
})
const confirmDialog = openModal.componentInstance as ConfirmDialogComponent
confirmDialog.confirmClicked.next(confirmDialog)
expect(closeSpy).toHaveBeenCalled()
expect(confirmDialog.messageBold).toContain('Document was updated at')
})
it('should react to websocket document updated notifications', () => {
initNormally()
const updateMessage = {
document_id: component.documentId,
modified: '2026-02-17T00:00:00Z',
owner_id: 1,
}
const handleSpy = jest
.spyOn(component as any, 'handleIncomingDocumentUpdated')
.mockImplementation(() => {})
const websocketStatusService = TestBed.inject(WebsocketStatusService)
websocketStatusService.handleDocumentUpdated(updateMessage)
expect(handleSpy).toHaveBeenCalledWith(updateMessage)
})
it('should queue incoming update while network is active and flush after', () => {
initNormally()
const loadSpy = jest.spyOn(component as any, 'loadDocument')
const toastSpy = jest.spyOn(toastService, 'showInfo')
component.networkActive = true
;(component as any).handleIncomingDocumentUpdated({
document_id: component.documentId,
modified: '2026-02-17T00:00:00Z',
})
expect(loadSpy).not.toHaveBeenCalled()
component.networkActive = false
;(component as any).flushPendingIncomingUpdate()
expect(loadSpy).toHaveBeenCalledWith(component.documentId, true)
expect(toastSpy).toHaveBeenCalledWith(
'Document reloaded with latest changes.'
)
})
it('should ignore queued incoming update matching local save modified', () => {
initNormally()
const loadSpy = jest.spyOn(component as any, 'loadDocument')
const toastSpy = jest.spyOn(toastService, 'showInfo')
component.networkActive = true
;(component as any).lastLocalSaveModified = '2026-02-17T00:00:00+00:00'
;(component as any).handleIncomingDocumentUpdated({
document_id: component.documentId,
modified: '2026-02-17T00:00:00+00:00',
})
component.networkActive = false
;(component as any).flushPendingIncomingUpdate()
expect(loadSpy).not.toHaveBeenCalled()
expect(toastSpy).not.toHaveBeenCalled()
})
it('should clear pdf source if preview URL is empty', () => {
component.pdfSource = { url: '/preview', password: 'secret' } as any
component.previewUrl = null
;(component as any).updatePdfSource()
expect(component.pdfSource).toEqual({ url: null, password: undefined })
})
it('should close incoming update modal if one is open', () => {
const modalRef = { close: jest.fn() } as unknown as NgbModalRef
;(component as any).incomingUpdateModal = modalRef
;(component as any).closeIncomingUpdateModal()
expect(modalRef.close).toHaveBeenCalled()
expect((component as any).incomingUpdateModal).toBeNull()
})
it('should reload remote version when incoming update modal is confirmed', async () => {
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modals) => (openModal = modals[0]))
const reloadSpy = jest
.spyOn(component as any, 'reloadRemoteVersion')
.mockImplementation(() => {})
;(component as any).showIncomingUpdateModal('2026-02-17T00:00:00Z')
const dialog = openModal.componentInstance as ConfirmDialogComponent
dialog.confirmClicked.next()
await openModal.result
expect(dialog.buttonsEnabled).toBe(false)
expect(reloadSpy).toHaveBeenCalled()
expect((component as any).incomingUpdateModal).toBeNull()
})
it('should overwrite open document state when loading remote version with force', () => {
const openDoc = Object.assign({}, doc, {
title: 'Locally edited title',
__changedFields: ['title'],
})
const remoteDoc = Object.assign({}, doc, {
title: 'Remote title',
modified: '2026-02-17T00:00:00Z',
})
jest.spyOn(documentService, 'get').mockReturnValue(of(remoteDoc))
jest.spyOn(documentService, 'getMetadata').mockReturnValue(
of({
has_archive_version: false,
original_mime_type: 'application/pdf',
})
)
jest.spyOn(documentService, 'getSuggestions').mockReturnValue(
of({
suggested_tags: [],
suggested_document_types: [],
suggested_correspondents: [],
})
)
jest.spyOn(openDocumentsService, 'getOpenDocument').mockReturnValue(openDoc)
const setDirtySpy = jest.spyOn(openDocumentsService, 'setDirty')
const saveSpy = jest.spyOn(openDocumentsService, 'save')
;(component as any).loadDocument(doc.id, true)
expect(openDoc.title).toEqual('Remote title')
expect(openDoc.__changedFields).toEqual([])
expect(setDirtySpy).toHaveBeenCalledWith(openDoc, false)
expect(saveSpy).toHaveBeenCalled()
})
it('should ignore incoming update for a different document id', () => {
initNormally()
const loadSpy = jest.spyOn(component as any, 'loadDocument')
;(component as any).handleIncomingDocumentUpdated({
document_id: component.documentId + 1,
modified: '2026-02-17T00:00:00Z',
})
expect(loadSpy).not.toHaveBeenCalled()
})
it('should show incoming update modal when local document has unsaved edits', () => {
initNormally()
jest.spyOn(openDocumentsService, 'isDirty').mockReturnValue(true)
const modalSpy = jest
.spyOn(component as any, 'showIncomingUpdateModal')
.mockImplementation(() => {})
;(component as any).handleIncomingDocumentUpdated({
document_id: component.documentId,
modified: '2026-02-17T00:00:00Z',
})
expect(modalSpy).toHaveBeenCalledWith('2026-02-17T00:00:00Z')
})
it('should reload current document and show toast when reloading remote version', () => {
component.documentId = doc.id
const closeModalSpy = jest
.spyOn(component as any, 'closeIncomingUpdateModal')
.mockImplementation(() => {})
const loadSpy = jest
.spyOn(component as any, 'loadDocument')
.mockImplementation(() => {})
const notifySpy = jest.spyOn(component.docChangeNotifier, 'next')
const toastSpy = jest.spyOn(toastService, 'showInfo')
;(component as any).reloadRemoteVersion()
expect(closeModalSpy).toHaveBeenCalled()
expect(notifySpy).toHaveBeenCalledWith(doc.id)
expect(loadSpy).toHaveBeenCalledWith(doc.id, true)
expect(toastSpy).toHaveBeenCalledWith('Document reloaded.')
})
it('should change preview element by render type', () => {
@@ -1721,6 +1923,14 @@ describe('DocumentDetailComponent', () => {
expect(component.createDisabled(DataType.Tag)).toBeFalsy()
})
it('should expose add permission via userCanAdd getter', () => {
currentUserCan = true
expect(component.userCanAdd).toBeTruthy()
currentUserCan = false
expect(component.userCanAdd).toBeFalsy()
})
it('should call tryRenderTiff when no archive and file is tiff', () => {
initNormally()
const tiffRenderSpy = jest.spyOn(

View File

@@ -13,6 +13,7 @@ import {
NgbDateStruct,
NgbDropdownModule,
NgbModal,
NgbModalRef,
NgbNav,
NgbNavChangeEvent,
NgbNavModule,
@@ -80,6 +81,7 @@ import { TagService } from 'src/app/services/rest/tag.service'
import { UserService } from 'src/app/services/rest/user.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { WebsocketStatusService } from 'src/app/services/websocket-status.service'
import { getFilenameFromContentDisposition } from 'src/app/utils/http'
import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
import * as UTIF from 'utif'
@@ -143,6 +145,11 @@ enum ContentRenderType {
TIFF = 'tiff',
}
interface IncomingDocumentUpdate {
document_id: number
modified: string
}
@Component({
selector: 'pngx-document-detail',
templateUrl: './document-detail.component.html',
@@ -208,6 +215,7 @@ export class DocumentDetailComponent
private componentRouterService = inject(ComponentRouterService)
private deviceDetectorService = inject(DeviceDetectorService)
private savedViewService = inject(SavedViewService)
private readonly websocketStatusService = inject(WebsocketStatusService)
@ViewChild('inputTitle')
titleInput: TextComponent
@@ -267,6 +275,9 @@ export class DocumentDetailComponent
isDirty$: Observable<boolean>
unsubscribeNotifier: Subject<any> = new Subject()
docChangeNotifier: Subject<any> = new Subject()
private incomingUpdateModal: NgbModalRef
private pendingIncomingUpdate: IncomingDocumentUpdate
private lastLocalSaveModified: string | null = null
requiresPassword: boolean = false
password: string
@@ -475,9 +486,12 @@ export class DocumentDetailComponent
)
}
private loadDocument(documentId: number): void {
private loadDocument(documentId: number, forceRemote: boolean = false): void {
let redirectedToRoot = false
this.closeIncomingUpdateModal()
this.pendingIncomingUpdate = null
this.selectedVersionId = documentId
this.lastLocalSaveModified = null
this.previewUrl = this.documentsService.getPreviewUrl(
this.selectedVersionId
)
@@ -545,21 +559,25 @@ export class DocumentDetailComponent
openDocument.duplicate_documents = doc.duplicate_documents
this.openDocumentService.save()
}
const useDoc = openDocument || doc
if (openDocument) {
if (
new Date(doc.modified) > new Date(openDocument.modified) &&
!this.modalService.hasOpenModals()
) {
const modal = this.modalService.open(ConfirmDialogComponent)
modal.componentInstance.title = $localize`Document changes detected`
modal.componentInstance.messageBold = $localize`The version of this document in your browser session appears older than the existing version.`
modal.componentInstance.message = $localize`Saving the document here may overwrite other changes that were made. To restore the existing version, discard your changes or close the document.`
modal.componentInstance.cancelBtnClass = 'visually-hidden'
modal.componentInstance.btnCaption = $localize`Ok`
modal.componentInstance.confirmClicked.subscribe(() =>
modal.close()
)
let useDoc = openDocument || doc
if (openDocument && forceRemote) {
Object.assign(openDocument, doc)
openDocument.__changedFields = []
this.openDocumentService.setDirty(openDocument, false)
this.openDocumentService.save()
useDoc = openDocument
} else if (openDocument) {
if (new Date(doc.modified) > new Date(openDocument.modified)) {
if (this.hasLocalEdits(openDocument)) {
this.showIncomingUpdateModal(doc.modified)
} else {
// No local edits to preserve, so keep the tab in sync automatically.
Object.assign(openDocument, doc)
openDocument.__changedFields = []
this.openDocumentService.setDirty(openDocument, false)
this.openDocumentService.save()
useDoc = openDocument
}
}
} else {
this.openDocumentService
@@ -590,6 +608,98 @@ export class DocumentDetailComponent
})
}
private hasLocalEdits(doc: Document): boolean {
return (
this.openDocumentService.isDirty(doc) || !!doc.__changedFields?.length
)
}
private showIncomingUpdateModal(modified: string): void {
if (this.incomingUpdateModal) return
const modal = this.modalService.open(ConfirmDialogComponent, {
backdrop: 'static',
})
this.incomingUpdateModal = modal
let formattedModified = null
const parsed = new Date(modified)
formattedModified = parsed.toLocaleString()
modal.componentInstance.title = $localize`Document was updated`
modal.componentInstance.messageBold = $localize`Document was updated at ${formattedModified}.`
modal.componentInstance.message = $localize`Reload to discard your local unsaved edits and load the latest remote version.`
modal.componentInstance.btnClass = 'btn-warning'
modal.componentInstance.btnCaption = $localize`Reload`
modal.componentInstance.cancelBtnCaption = $localize`Dismiss`
modal.componentInstance.confirmClicked.pipe(first()).subscribe(() => {
modal.componentInstance.buttonsEnabled = false
modal.close()
this.reloadRemoteVersion()
})
modal.result.finally(() => {
this.incomingUpdateModal = null
})
}
private closeIncomingUpdateModal() {
if (!this.incomingUpdateModal) return
this.incomingUpdateModal.close()
this.incomingUpdateModal = null
}
private flushPendingIncomingUpdate() {
if (!this.pendingIncomingUpdate || this.networkActive) return
const pendingUpdate = this.pendingIncomingUpdate
this.pendingIncomingUpdate = null
this.handleIncomingDocumentUpdated(pendingUpdate)
}
private handleIncomingDocumentUpdated(data: IncomingDocumentUpdate): void {
if (
!this.documentId ||
!this.document ||
data.document_id !== this.documentId
)
return
if (this.networkActive) {
this.pendingIncomingUpdate = data
return
}
// If modified timestamp of the incoming update is the same as the last local save,
// we assume this update is from our own save and dont notify
const incomingModified = data.modified
if (
incomingModified &&
this.lastLocalSaveModified &&
incomingModified === this.lastLocalSaveModified
) {
this.lastLocalSaveModified = null
return
}
this.lastLocalSaveModified = null
if (this.openDocumentService.isDirty(this.document)) {
this.showIncomingUpdateModal(data.modified)
} else {
this.docChangeNotifier.next(this.documentId)
this.loadDocument(this.documentId, true)
this.toastService.showInfo(
$localize`Document reloaded with latest changes.`
)
}
}
private reloadRemoteVersion() {
if (!this.documentId) return
this.closeIncomingUpdateModal()
this.docChangeNotifier.next(this.documentId)
this.loadDocument(this.documentId, true)
this.toastService.showInfo($localize`Document reloaded.`)
}
ngOnInit(): void {
this.setZoom(
this.settings.get(SETTINGS_KEYS.PDF_VIEWER_ZOOM_SETTING) as PdfZoomScale
@@ -648,6 +758,11 @@ export class DocumentDetailComponent
this.getCustomFields()
this.websocketStatusService
.onDocumentUpdated()
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe((data) => this.handleIncomingDocumentUpdated(data))
this.route.paramMap
.pipe(
filter(
@@ -1033,6 +1148,7 @@ export class DocumentDetailComponent
)
.subscribe({
next: (doc) => {
this.closeIncomingUpdateModal()
Object.assign(this.document, doc)
doc['permissions_form'] = {
owner: doc.owner,
@@ -1079,6 +1195,8 @@ export class DocumentDetailComponent
.pipe(first())
.subscribe({
next: (docValues) => {
this.closeIncomingUpdateModal()
this.lastLocalSaveModified = docValues.modified ?? null
// in case data changed while saving eg removing inbox_tags
this.documentForm.patchValue(docValues)
const newValues = Object.assign({}, this.documentForm.value)
@@ -1093,16 +1211,19 @@ export class DocumentDetailComponent
this.networkActive = false
this.error = null
if (close) {
this.pendingIncomingUpdate = null
this.close(() =>
this.openDocumentService.refreshDocument(this.documentId)
)
} else {
this.openDocumentService.refreshDocument(this.documentId)
this.flushPendingIncomingUpdate()
}
this.savedViewService.maybeRefreshDocumentCounts()
},
error: (error) => {
this.networkActive = false
this.lastLocalSaveModified = null
const canEdit =
this.permissionsService.currentUserHasObjectPermissions(
PermissionAction.Change,
@@ -1122,6 +1243,7 @@ export class DocumentDetailComponent
error
)
}
this.flushPendingIncomingUpdate()
},
})
}
@@ -1158,8 +1280,11 @@ export class DocumentDetailComponent
.pipe(first())
.subscribe({
next: ({ updateResult, nextDocId, closeResult }) => {
this.closeIncomingUpdateModal()
this.error = null
this.networkActive = false
this.pendingIncomingUpdate = null
this.lastLocalSaveModified = null
if (closeResult && updateResult && nextDocId) {
this.router.navigate(['documents', nextDocId])
this.titleInput?.focus()
@@ -1167,8 +1292,10 @@ export class DocumentDetailComponent
},
error: (error) => {
this.networkActive = false
this.lastLocalSaveModified = null
this.error = error.error
this.toastService.showError($localize`Error saving document`, error)
this.flushPendingIncomingUpdate()
},
})
}
@@ -1254,7 +1381,7 @@ export class DocumentDetailComponent
.subscribe({
next: () => {
this.toastService.showInfo(
$localize`Reprocess operation for "${this.document.title}" will begin in the background. Close and re-open or reload this document after the operation has completed to see new content.`
$localize`Reprocess operation for "${this.document.title}" will begin in the background.`
)
if (modal) {
modal.close()

View File

@@ -57,7 +57,7 @@
}
</div>
@for (version of versions; track version.id) {
<div class="dropdown-item border-top px-0">
<div class="dropdown-item border-top px-0" [class.pe-3]="versions.length === 1">
<div class="d-flex align-items-center w-100 py-2 version-item">
<div class="btn btn-link link-underline link-underline-opacity-0 d-flex align-items-center small text-start p-0 version-link"
(click)="selectVersion(version.id)"
@@ -88,7 +88,7 @@
@if (version.version_label) {
{{ version.version_label }}
} @else {
<span i18n>Version</span>&nbsp;#{{ version.id }}
<span class="fst-italic"><ng-container i18n>Version</ng-container>&nbsp;{{ versions.length - $index }}&nbsp;<span class="text-muted small">(#{{ version.id }})</span></span>
}
</span>
}

View File

@@ -830,7 +830,7 @@ export class BulkEditorComponent
})
const rotateDialog = modal.componentInstance as RotateConfirmDialogComponent
rotateDialog.title = $localize`Rotate confirm`
rotateDialog.messageBold = $localize`This operation will permanently rotate the original version of ${this.list.selected.size} document(s).`
rotateDialog.messageBold = $localize`This operation will add rotated versions of the ${this.list.selected.size} document(s).`
rotateDialog.btnClass = 'btn-danger'
rotateDialog.btnCaption = $localize`Proceed`
rotateDialog.documentID = Array.from(this.list.selected)[0]

View File

@@ -104,11 +104,9 @@
}
}
<div *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.SavedView }">
@if (list.activeSavedViewId) {
<button ngbDropdownItem (click)="saveViewConfig()" [disabled]="!savedViewIsModified" i18n>Save "{{list.activeSavedViewTitle}}"</button>
}
</div>
@if (list.activeSavedViewId && activeSavedViewCanChange) {
<button ngbDropdownItem (click)="saveViewConfig()" [disabled]="!savedViewIsModified" i18n>Save "{{list.activeSavedViewTitle}}"</button>
}
<button ngbDropdownItem (click)="saveViewConfigAs()" *pngxIfPermissions="{ action: PermissionAction.Add, type: PermissionType.SavedView }" i18n>Save as...</button>
<a ngbDropdownItem routerLink="/savedviews" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }" i18n>All saved views</a>
</div>

View File

@@ -168,6 +168,10 @@ describe('DocumentListComponent', () => {
)
})
it('should not allow changing a saved view when none is active', () => {
expect(component.activeSavedViewCanChange).toBeFalsy()
})
it('should determine if filtered, support reset', () => {
fixture.detectChanges()
documentListService.setFilterRules([
@@ -299,6 +303,19 @@ describe('DocumentListComponent', () => {
expect(setCountSpy).toHaveBeenCalledWith(expect.any(Object), 3)
})
it('should reset active saved view when loading unknown view config', () => {
component['activeSavedView'] = { id: 1 } as SavedView
const activateSpy = jest.spyOn(documentListService, 'activateSavedView')
const reloadSpy = jest.spyOn(documentListService, 'reload')
jest.spyOn(savedViewService, 'getCached').mockReturnValue(of(null))
component.loadViewConfig(10)
expect(component['activeSavedView']).toBeNull()
expect(activateSpy).not.toHaveBeenCalled()
expect(reloadSpy).not.toHaveBeenCalled()
})
it('should support 3 different display modes', () => {
jest.spyOn(documentListService, 'documents', 'get').mockReturnValue(docs)
fixture.detectChanges()
@@ -466,7 +483,7 @@ describe('DocumentListComponent', () => {
})
it('should handle error on view saving', () => {
component.list.activateSavedView({
const view: SavedView = {
id: 10,
name: 'Saved View 10',
sort_field: 'added',
@@ -477,7 +494,16 @@ describe('DocumentListComponent', () => {
value: '20',
},
],
})
}
jest.spyOn(savedViewService, 'getCached').mockReturnValue(of(view))
const queryParams = { view: view.id.toString() }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
activatedRoute.snapshot.queryParams = queryParams
router.routerState.snapshot.url = '/view/10/'
fixture.detectChanges()
const toastErrorSpy = jest.spyOn(toastService, 'showError')
jest
.spyOn(savedViewService, 'patch')
@@ -489,6 +515,40 @@ describe('DocumentListComponent', () => {
)
})
it('should not save a view without object change permissions', () => {
const view: SavedView = {
id: 10,
name: 'Saved View 10',
sort_field: 'added',
sort_reverse: true,
filter_rules: [
{
rule_type: FILTER_HAS_TAGS_ANY,
value: '20',
},
],
owner: 999,
user_can_change: false,
}
jest.spyOn(savedViewService, 'getCached').mockReturnValue(of(view))
jest
.spyOn(permissionService, 'currentUserHasObjectPermissions')
.mockReturnValue(false)
const queryParams = { view: view.id.toString() }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
activatedRoute.snapshot.queryParams = queryParams
router.routerState.snapshot.url = '/view/10/'
fixture.detectChanges()
const patchSpy = jest.spyOn(savedViewService, 'patch')
component.saveViewConfig()
expect(patchSpy).not.toHaveBeenCalled()
})
it('should support edited view saving as', () => {
const view: SavedView = {
id: 10,
@@ -520,19 +580,105 @@ describe('DocumentListComponent', () => {
const modalSpy = jest.spyOn(modalService, 'open')
const toastSpy = jest.spyOn(toastService, 'showInfo')
const savedViewServiceCreate = jest.spyOn(savedViewService, 'create')
jest
.spyOn(savedViewService, 'dashboardViews', 'get')
.mockReturnValue([{ id: 77 } as SavedView])
jest
.spyOn(savedViewService, 'sidebarViews', 'get')
.mockReturnValue([{ id: 88 } as SavedView])
const updateVisibilitySpy = jest
.spyOn(settingsService, 'updateSavedViewsVisibility')
.mockReturnValue(of({ success: true }))
savedViewServiceCreate.mockReturnValueOnce(of(modifiedView))
component.saveViewConfigAs()
const modalCloseSpy = jest.spyOn(openModal, 'close')
const permissions = {
owner: 5,
set_permissions: {
view: {
users: [4],
groups: [3],
},
change: {
users: [2],
groups: [1],
},
},
}
openModal.componentInstance.saveClicked.next({
name: 'Foo Bar',
showOnDashboard: true,
showInSideBar: true,
permissions_form: permissions,
})
expect(savedViewServiceCreate).toHaveBeenCalled()
expect(savedViewServiceCreate).toHaveBeenCalledWith(
expect.objectContaining({
name: 'Foo Bar',
owner: permissions.owner,
set_permissions: permissions.set_permissions,
})
)
expect(updateVisibilitySpy).toHaveBeenCalledWith(
expect.arrayContaining([77, modifiedView.id]),
expect.arrayContaining([88, modifiedView.id])
)
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
})
it('should show error when visibility update fails after creating a view', () => {
const view: SavedView = {
id: 10,
name: 'Saved View 10',
sort_field: 'added',
sort_reverse: true,
filter_rules: [
{
rule_type: FILTER_HAS_TAGS_ANY,
value: '20',
},
],
}
jest.spyOn(savedViewService, 'getCached').mockReturnValue(of(view))
const queryParams = { view: view.id.toString() }
jest
.spyOn(activatedRoute, 'queryParamMap', 'get')
.mockReturnValue(of(convertToParamMap(queryParams)))
activatedRoute.snapshot.queryParams = queryParams
router.routerState.snapshot.url = '/view/10/'
fixture.detectChanges()
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
jest
.spyOn(savedViewService, 'create')
.mockReturnValueOnce(of({ ...view, id: 42, name: 'Foo Bar' }))
jest.spyOn(savedViewService, 'dashboardViews', 'get').mockReturnValue([])
jest.spyOn(savedViewService, 'sidebarViews', 'get').mockReturnValue([])
jest
.spyOn(settingsService, 'updateSavedViewsVisibility')
.mockReturnValueOnce(
throwError(() => new Error('unable to save visibility settings'))
)
const toastErrorSpy = jest.spyOn(toastService, 'showError')
component.saveViewConfigAs()
const modalCloseSpy = jest.spyOn(openModal, 'close')
openModal.componentInstance.saveClicked.next({
name: 'Foo Bar',
show_on_dashboard: true,
show_in_sidebar: true,
showOnDashboard: true,
showInSideBar: false,
})
expect(savedViewServiceCreate).toHaveBeenCalled()
expect(modalSpy).toHaveBeenCalled()
expect(toastSpy).toHaveBeenCalled()
expect(modalCloseSpy).toHaveBeenCalled()
expect(toastErrorSpy).toHaveBeenCalledWith(
'View "Foo Bar" created successfully, but could not update visibility settings.',
expect.any(Error)
)
})
it('should handle error on edited view saving as', () => {
@@ -563,6 +709,10 @@ describe('DocumentListComponent', () => {
let openModal: NgbModalRef
modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
const updateVisibilitySpy = jest.spyOn(
settingsService,
'updateSavedViewsVisibility'
)
jest.spyOn(savedViewService, 'create').mockReturnValueOnce(
throwError(
() =>
@@ -575,9 +725,10 @@ describe('DocumentListComponent', () => {
openModal.componentInstance.saveClicked.next({
name: 'Foo Bar',
show_on_dashboard: true,
show_in_sidebar: true,
showOnDashboard: true,
showInSideBar: true,
})
expect(updateVisibilitySpy).not.toHaveBeenCalled()
expect(openModal.componentInstance.error).toEqual({ filter_rules: ['11'] })
})

View File

@@ -47,7 +47,10 @@ import { UsernamePipe } from 'src/app/pipes/username.pipe'
import { DocumentListViewService } from 'src/app/services/document-list-view.service'
import { HotKeyService } from 'src/app/services/hot-key.service'
import { OpenDocumentsService } from 'src/app/services/open-documents.service'
import { PermissionsService } from 'src/app/services/permissions.service'
import {
PermissionAction,
PermissionsService,
} from 'src/app/services/permissions.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
@@ -148,12 +151,18 @@ export class DocumentListComponent
unmodifiedFilterRules: FilterRule[] = []
private unmodifiedSavedView: SavedView
private activeSavedView: SavedView | null = null
private unsubscribeNotifier: Subject<any> = new Subject()
get savedViewIsModified(): boolean {
if (!this.list.activeSavedViewId || !this.unmodifiedSavedView) return false
else {
if (
!this.list.activeSavedViewId ||
!this.unmodifiedSavedView ||
!this.activeSavedViewCanChange
) {
return false
} else {
return (
this.unmodifiedSavedView.sort_field !== this.list.sortField ||
this.unmodifiedSavedView.sort_reverse !== this.list.sortReverse ||
@@ -180,6 +189,16 @@ export class DocumentListComponent
}
}
get activeSavedViewCanChange(): boolean {
if (!this.activeSavedView) {
return false
}
return this.permissionService.currentUserHasObjectPermissions(
PermissionAction.Change,
this.activeSavedView
)
}
get isFiltered() {
return !!this.filterEditor?.rulesModified
}
@@ -264,11 +283,13 @@ export class DocumentListComponent
.pipe(takeUntil(this.unsubscribeNotifier))
.subscribe(({ view }) => {
if (!view) {
this.activeSavedView = null
this.router.navigate(['404'], {
replaceUrl: true,
})
return
}
this.activeSavedView = view
this.unmodifiedSavedView = view
this.list.activateSavedViewWithQueryParams(
view,
@@ -292,6 +313,7 @@ export class DocumentListComponent
// loading a saved view on /documents
this.loadViewConfig(parseInt(queryParams.get('view')))
} else {
this.activeSavedView = null
this.list.activateSavedView(null)
this.list.loadFromQueryParams(queryParams)
this.unmodifiedFilterRules = []
@@ -374,7 +396,7 @@ export class DocumentListComponent
}
saveViewConfig() {
if (this.list.activeSavedViewId != null) {
if (this.list.activeSavedViewId != null && this.activeSavedViewCanChange) {
let savedView: SavedView = {
id: this.list.activeSavedViewId,
filter_rules: this.list.filterRules,
@@ -388,6 +410,7 @@ export class DocumentListComponent
.pipe(first())
.subscribe({
next: (view) => {
this.activeSavedView = view
this.unmodifiedSavedView = view
this.toastService.showInfo(
$localize`View "${this.list.activeSavedViewTitle}" saved successfully.`
@@ -409,6 +432,11 @@ export class DocumentListComponent
.getCached(viewID)
.pipe(first())
.subscribe((view) => {
if (!view) {
this.activeSavedView = null
return
}
this.activeSavedView = view
this.unmodifiedSavedView = view
this.list.activateSavedView(view)
this.list.reload(() => {
@@ -426,24 +454,48 @@ export class DocumentListComponent
modal.componentInstance.buttonsEnabled = false
let savedView: SavedView = {
name: formValue.name,
show_on_dashboard: formValue.showOnDashboard,
show_in_sidebar: formValue.showInSideBar,
filter_rules: this.list.filterRules,
sort_reverse: this.list.sortReverse,
sort_field: this.list.sortField,
display_mode: this.list.displayMode,
display_fields: this.activeDisplayFields,
}
const permissions = formValue.permissions_form
if (permissions) {
if (permissions.owner !== null && permissions.owner !== undefined) {
savedView.owner = permissions.owner
}
if (permissions.set_permissions) {
savedView['set_permissions'] = permissions.set_permissions
}
}
this.savedViewService
.create(savedView)
.pipe(first())
.subscribe({
next: () => {
modal.close()
this.toastService.showInfo(
$localize`View "${savedView.name}" created successfully.`
next: (createdView) => {
this.saveCreatedViewVisibility(
createdView,
formValue.showOnDashboard,
formValue.showInSideBar
)
.pipe(first())
.subscribe({
next: () => {
modal.close()
this.toastService.showInfo(
$localize`View "${savedView.name}" created successfully.`
)
},
error: (error) => {
modal.close()
this.toastService.showError(
$localize`View "${savedView.name}" created successfully, but could not update visibility settings.`,
error
)
},
})
},
error: (httpError) => {
let error = httpError.error
@@ -457,6 +509,28 @@ export class DocumentListComponent
})
}
private saveCreatedViewVisibility(
createdView: SavedView,
showOnDashboard: boolean,
showInSideBar: boolean
) {
const dashboardViewIds = this.savedViewService.dashboardViews.map(
(v) => v.id
)
const sidebarViewIds = this.savedViewService.sidebarViews.map((v) => v.id)
if (showOnDashboard) {
dashboardViewIds.push(createdView.id)
}
if (showInSideBar) {
sidebarViewIds.push(createdView.id)
}
return this.settingsService.updateSavedViewsVisibility(
dashboardViewIds,
sidebarViewIds
)
}
openDocumentDetail(document: Document | number) {
this.router.navigate([
'documents',

View File

@@ -8,6 +8,7 @@
<pngx-input-text i18n-title title="Name" formControlName="name" [error]="error?.name" autocomplete="off"></pngx-input-text>
<pngx-input-check i18n-title title="Show in sidebar" formControlName="showInSideBar"></pngx-input-check>
<pngx-input-check i18n-title title="Show on dashboard" formControlName="showOnDashboard"></pngx-input-check>
<pngx-permissions-form accordion="true" formControlName="permissions_form"></pngx-permissions-form>
@if (error?.filter_rules) {
<div class="alert alert-danger" role="alert">
<h6 class="alert-heading" i18n>Filter rules error occurred while saving this view</h6>

View File

@@ -7,7 +7,13 @@ import {
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { By } from '@angular/platform-browser'
import { NgbActiveModal, NgbModalModule } from '@ng-bootstrap/ng-bootstrap'
import { of } from 'rxjs'
import { GroupService } from 'src/app/services/rest/group.service'
import { UserService } from 'src/app/services/rest/user.service'
import { CheckComponent } from '../../common/input/check/check.component'
import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
import { PermissionsGroupComponent } from '../../common/input/permissions/permissions-group/permissions-group.component'
import { PermissionsUserComponent } from '../../common/input/permissions/permissions-user/permissions-user.component'
import { TextComponent } from '../../common/input/text/text.component'
import { SaveViewConfigDialogComponent } from './save-view-config-dialog.component'
@@ -18,7 +24,21 @@ describe('SaveViewConfigDialogComponent', () => {
beforeEach(fakeAsync(() => {
TestBed.configureTestingModule({
providers: [NgbActiveModal],
providers: [
NgbActiveModal,
{
provide: UserService,
useValue: {
listAll: () => of({ results: [] }),
},
},
{
provide: GroupService,
useValue: {
listAll: () => of({ results: [] }),
},
},
],
imports: [
NgbModalModule,
FormsModule,
@@ -26,6 +46,9 @@ describe('SaveViewConfigDialogComponent', () => {
SaveViewConfigDialogComponent,
TextComponent,
CheckComponent,
PermissionsFormComponent,
PermissionsUserComponent,
PermissionsGroupComponent,
],
}).compileComponents()
@@ -81,6 +104,26 @@ describe('SaveViewConfigDialogComponent', () => {
})
})
it('should support permissions input', () => {
const permissions = {
owner: 10,
set_permissions: {
view: { users: [2], groups: [3] },
change: { users: [4], groups: [5] },
},
}
let result
component.saveClicked.subscribe((saveResult) => (result = saveResult))
component.saveViewConfigForm.get('permissions_form').patchValue(permissions)
component.save()
expect(result).toEqual({
name: '',
showInSideBar: false,
showOnDashboard: false,
permissions_form: permissions,
})
})
it('should support default name', () => {
const saveClickedSpy = jest.spyOn(component.saveClicked, 'emit')
const modalCloseSpy = jest.spyOn(modal, 'close')

View File

@@ -13,14 +13,22 @@ import {
ReactiveFormsModule,
} from '@angular/forms'
import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
import { User } from 'src/app/data/user'
import { CheckComponent } from '../../common/input/check/check.component'
import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
import { TextComponent } from '../../common/input/text/text.component'
@Component({
selector: 'pngx-save-view-config-dialog',
templateUrl: './save-view-config-dialog.component.html',
styleUrls: ['./save-view-config-dialog.component.scss'],
imports: [CheckComponent, TextComponent, FormsModule, ReactiveFormsModule],
imports: [
CheckComponent,
TextComponent,
PermissionsFormComponent,
FormsModule,
ReactiveFormsModule,
],
})
export class SaveViewConfigDialogComponent implements OnInit {
private modal = inject(NgbActiveModal)
@@ -36,6 +44,8 @@ export class SaveViewConfigDialogComponent implements OnInit {
closeEnabled = false
users: User[]
_defaultName = ''
get defaultName() {
@@ -52,6 +62,7 @@ export class SaveViewConfigDialogComponent implements OnInit {
name: new FormControl(''),
showInSideBar: new FormControl(false),
showOnDashboard: new FormControl(false),
permissions_form: new FormControl(null),
})
ngOnInit(): void {
@@ -62,7 +73,16 @@ export class SaveViewConfigDialogComponent implements OnInit {
}
save() {
this.saveClicked.emit(this.saveViewConfigForm.value)
const formValue = this.saveViewConfigForm.value
const saveViewConfig = {
name: formValue.name,
showInSideBar: formValue.showInSideBar,
showOnDashboard: formValue.showOnDashboard,
}
if (formValue.permissions_form) {
saveViewConfig['permissions_form'] = formValue.permissions_form
}
this.saveClicked.emit(saveViewConfig)
}
cancel() {

View File

@@ -25,15 +25,23 @@
</div>
</div>
<div class="col-auto">
<label class="form-label" for="name_{{view.id}}" i18n>Actions</label>
<pngx-confirm-button
label="Delete"
i18n-label
(confirm)="deleteSavedView(view)"
*pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.SavedView }"
buttonClasses="btn-sm btn-outline-danger form-control"
iconName="trash">
</pngx-confirm-button>
@if (canDeleteSavedView(view)) {
<label class="form-label" for="name_{{view.id}}" i18n>Actions</label>
<button
class="btn btn-sm btn-outline-secondary form-control mb-2"
type="button"
(click)="editPermissions(view)"
*pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.SavedView }"
i18n><i-bs class="me-1" name="person-fill-lock"></i-bs>Permissions</button>
<pngx-confirm-button
label="Delete"
i18n-label
(confirm)="deleteSavedView(view)"
*pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.SavedView }"
buttonClasses="btn-sm btn-outline-danger form-control"
iconName="trash">
</pngx-confirm-button>
}
</div>
</div>
<div class="row">

View File

@@ -3,16 +3,16 @@ import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
import { provideHttpClientTesting } from '@angular/common/http/testing'
import { ComponentFixture, TestBed } from '@angular/core/testing'
import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { By } from '@angular/platform-browser'
import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgbModal, NgbModule } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
import { of, throwError } from 'rxjs'
import { Subject, of, throwError } from 'rxjs'
import { SavedView } from 'src/app/data/saved-view'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import { PermissionsGuard } from 'src/app/guards/permissions.guard'
import { PermissionsService } from 'src/app/services/permissions.service'
import { CustomFieldsService } from 'src/app/services/rest/custom-fields.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component'
import { CheckComponent } from '../../common/input/check/check.component'
@@ -32,7 +32,9 @@ describe('SavedViewsComponent', () => {
let component: SavedViewsComponent
let fixture: ComponentFixture<SavedViewsComponent>
let savedViewService: SavedViewService
let settingsService: SettingsService
let toastService: ToastService
let modalService: NgbModal
beforeEach(async () => {
TestBed.configureTestingModule({
@@ -57,6 +59,8 @@ describe('SavedViewsComponent', () => {
provide: PermissionsService,
useValue: {
currentUserCan: () => true,
currentUserHasObjectPermissions: () => true,
currentUserOwnsObject: () => true,
},
},
{
@@ -77,11 +81,13 @@ describe('SavedViewsComponent', () => {
}).compileComponents()
savedViewService = TestBed.inject(SavedViewService)
settingsService = TestBed.inject(SettingsService)
toastService = TestBed.inject(ToastService)
modalService = TestBed.inject(NgbModal)
fixture = TestBed.createComponent(SavedViewsComponent)
component = fixture.componentInstance
jest.spyOn(savedViewService, 'listAll').mockReturnValue(
jest.spyOn(savedViewService, 'list').mockReturnValue(
of({
all: savedViews.map((v) => v.id),
count: savedViews.length,
@@ -94,14 +100,13 @@ describe('SavedViewsComponent', () => {
it('should support save saved views, show error', () => {
const toastErrorSpy = jest.spyOn(toastService, 'showError')
const toastSpy = jest.spyOn(toastService, 'show')
const savedViewPatchSpy = jest.spyOn(savedViewService, 'patchMany')
const toggle = fixture.debugElement.query(
By.css('.form-check.form-switch input')
)
toggle.nativeElement.checked = true
toggle.nativeElement.dispatchEvent(new Event('change'))
const control = component.savedViewsForm
.get('savedViews')
.get(savedViews[0].id.toString())
.get('name')
control.setValue(`${savedViews[0].name}-changed`)
control.markAsDirty()
// saved views error first
savedViewPatchSpy.mockReturnValueOnce(
@@ -110,12 +115,13 @@ describe('SavedViewsComponent', () => {
component.save()
expect(toastErrorSpy).toHaveBeenCalled()
expect(savedViewPatchSpy).toHaveBeenCalled()
toastSpy.mockClear()
toastErrorSpy.mockClear()
savedViewPatchSpy.mockClear()
// succeed saved views
savedViewPatchSpy.mockReturnValueOnce(of(savedViews as SavedView[]))
control.setValue(savedViews[0].name)
control.markAsDirty()
component.save()
expect(toastErrorSpy).not.toHaveBeenCalled()
expect(savedViewPatchSpy).toHaveBeenCalled()
@@ -127,26 +133,65 @@ describe('SavedViewsComponent', () => {
expect(patchSpy).not.toHaveBeenCalled()
const view = savedViews[0]
const toggle = fixture.debugElement.query(
By.css('.form-check.form-switch input')
)
toggle.nativeElement.checked = true
toggle.nativeElement.dispatchEvent(new Event('change'))
// register change
component.savedViewsForm.get('savedViews').get(view.id.toString()).value[
'show_on_dashboard'
] = !view.show_on_dashboard
component.savedViewsForm
.get('savedViews')
.get(view.id.toString())
.get('name')
.setValue('changed-view-name')
component.savedViewsForm
.get('savedViews')
.get(view.id.toString())
.get('name')
.markAsDirty()
fixture.detectChanges()
component.save()
expect(patchSpy).toHaveBeenCalledWith([
{
id: view.id,
name: view.name,
show_in_sidebar: view.show_in_sidebar,
show_on_dashboard: !view.show_on_dashboard,
},
])
expect(patchSpy).toHaveBeenCalled()
const patchBody = patchSpy.mock.calls[0][0][0]
expect(patchBody).toMatchObject({
id: view.id,
name: 'changed-view-name',
})
expect(patchBody.show_on_dashboard).toBeUndefined()
expect(patchBody.show_in_sidebar).toBeUndefined()
})
it('should persist visibility changes to user settings', () => {
const patchSpy = jest.spyOn(savedViewService, 'patchMany')
const updateVisibilitySpy = jest
.spyOn(settingsService, 'updateSavedViewsVisibility')
.mockReturnValue(of({ success: true }))
const dashboardControl = component.savedViewsForm
.get('savedViews')
.get(savedViews[0].id.toString())
.get('show_on_dashboard')
dashboardControl.setValue(false)
dashboardControl.markAsDirty()
component.save()
expect(patchSpy).not.toHaveBeenCalled()
expect(updateVisibilitySpy).toHaveBeenCalledWith([], [savedViews[0].id])
})
it('should skip model updates for views that cannot be edited', () => {
const patchSpy = jest.spyOn(savedViewService, 'patchMany')
const updateVisibilitySpy = jest.spyOn(
settingsService,
'updateSavedViewsVisibility'
)
const nameControl = component.savedViewsForm
.get('savedViews')
.get(savedViews[0].id.toString())
.get('name')
nameControl.disable()
component.save()
expect(patchSpy).not.toHaveBeenCalled()
expect(updateVisibilitySpy).not.toHaveBeenCalled()
})
it('should support delete saved view', () => {
@@ -162,14 +207,55 @@ describe('SavedViewsComponent', () => {
it('should support reset', () => {
const view = savedViews[0]
component.savedViewsForm.get('savedViews').get(view.id.toString()).value[
'show_on_dashboard'
] = !view.show_on_dashboard
component.savedViewsForm
.get('savedViews')
.get(view.id.toString())
.get('show_on_dashboard')
.setValue(!view.show_on_dashboard)
component.reset()
expect(
component.savedViewsForm.get('savedViews').get(view.id.toString()).value[
'show_on_dashboard'
]
component.savedViewsForm
.get('savedViews')
.get(view.id.toString())
.get('show_on_dashboard').value
).toEqual(view.show_on_dashboard)
})
it('should support editing permissions', () => {
const confirmClicked = new Subject<any>()
const modalRef = {
componentInstance: {
confirmClicked,
buttonsEnabled: true,
},
close: jest.fn(),
} as any
jest.spyOn(modalService, 'open').mockReturnValue(modalRef)
const patchSpy = jest.spyOn(savedViewService, 'patch')
patchSpy.mockReturnValue(of(savedViews[0] as SavedView))
component.editPermissions(savedViews[0] as SavedView)
confirmClicked.next({
permissions: {
owner: 1,
set_permissions: {
view: { users: [2], groups: [] },
change: { users: [], groups: [3] },
},
},
merge: true,
})
expect(patchSpy).toHaveBeenCalledWith(
expect.objectContaining({
id: savedViews[0].id,
owner: 1,
set_permissions: {
view: { users: [2], groups: [] },
change: { users: [], groups: [3] },
},
})
)
expect(modalRef.close).toHaveBeenCalled()
})
})

View File

@@ -6,11 +6,18 @@ import {
FormsModule,
ReactiveFormsModule,
} from '@angular/forms'
import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
import { dirtyCheck } from '@ngneat/dirty-check-forms'
import { BehaviorSubject, Observable, takeUntil } from 'rxjs'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
import { BehaviorSubject, Observable, of, switchMap, takeUntil } from 'rxjs'
import { PermissionsDialogComponent } from 'src/app/components/common/permissions-dialog/permissions-dialog.component'
import { DisplayMode } from 'src/app/data/document'
import { SavedView } from 'src/app/data/saved-view'
import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
import {
PermissionAction,
PermissionsService,
} from 'src/app/services/permissions.service'
import { SavedViewService } from 'src/app/services/rest/saved-view.service'
import { SettingsService } from 'src/app/services/settings.service'
import { ToastService } from 'src/app/services/toast.service'
@@ -34,15 +41,18 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
FormsModule,
ReactiveFormsModule,
AsyncPipe,
NgxBootstrapIconsModule,
],
})
export class SavedViewsComponent
extends LoadingComponentWithPermissions
implements OnInit, OnDestroy
{
private savedViewService = inject(SavedViewService)
private settings = inject(SettingsService)
private toastService = inject(ToastService)
private readonly savedViewService = inject(SavedViewService)
private readonly permissionsService = inject(PermissionsService)
private readonly settings = inject(SettingsService)
private readonly toastService = inject(ToastService)
private readonly modalService = inject(NgbModal)
DisplayMode = DisplayMode
@@ -65,11 +75,17 @@ export class SavedViewsComponent
}
ngOnInit(): void {
this.reloadViews()
}
private reloadViews(): void {
this.loading = true
this.savedViewService.listAll().subscribe((r) => {
this.savedViews = r.results
this.initialize()
})
this.savedViewService
.list(null, null, null, false, { full_perms: true })
.subscribe((r) => {
this.savedViews = r.results
this.initialize()
})
}
ngOnDestroy(): void {
@@ -95,16 +111,20 @@ export class SavedViewsComponent
display_mode: view.display_mode,
display_fields: view.display_fields,
}
const canEdit = this.canEditSavedView(view)
this.savedViewsGroup.addControl(
view.id.toString(),
new FormGroup({
id: new FormControl(null),
name: new FormControl(null),
show_on_dashboard: new FormControl(null),
show_in_sidebar: new FormControl(null),
page_size: new FormControl(null),
display_mode: new FormControl(null),
display_fields: new FormControl([]),
id: new FormControl({ value: null, disabled: !canEdit }),
name: new FormControl({ value: null, disabled: !canEdit }),
show_on_dashboard: new FormControl({
value: null,
disabled: false,
}),
show_in_sidebar: new FormControl({ value: null, disabled: false }),
page_size: new FormControl({ value: null, disabled: !canEdit }),
display_mode: new FormControl({ value: null, disabled: !canEdit }),
display_fields: new FormControl({ value: [], disabled: !canEdit }),
})
)
}
@@ -133,10 +153,7 @@ export class SavedViewsComponent
$localize`Saved view "${savedView.name}" deleted.`
)
this.savedViewService.clearCache()
this.savedViewService.listAll().subscribe((r) => {
this.savedViews = r.results
this.initialize()
})
this.reloadViews()
})
}
@@ -145,26 +162,120 @@ export class SavedViewsComponent
}
public save() {
// only patch views that have actually changed
// Save only changed views, then save the visibility changes into user settings.
const groups = Object.values(this.savedViewsGroup.controls) as FormGroup[]
const visibilityChanged = groups.some(
(group) =>
group.get('show_on_dashboard')?.dirty ||
group.get('show_in_sidebar')?.dirty
)
const changed: SavedView[] = []
Object.values(this.savedViewsGroup.controls)
.filter((g: FormGroup) => !g.pristine)
.forEach((group: FormGroup) => {
changed.push(group.value)
})
const dashboardVisibleIds: number[] = []
const sidebarVisibleIds: number[] = []
groups.forEach((group) => {
const value = group.getRawValue()
if (value.show_on_dashboard) {
dashboardVisibleIds.push(value.id)
}
if (value.show_in_sidebar) {
sidebarVisibleIds.push(value.id)
}
// Would be fine to send, but no longer stored on the model
delete value.show_on_dashboard
delete value.show_in_sidebar
if (!group.get('name')?.enabled) {
// Quick check for user doesn't have permissions, then bail
return
}
const modelFieldsChanged =
group.get('name')?.dirty ||
group.get('page_size')?.dirty ||
group.get('display_mode')?.dirty ||
group.get('display_fields')?.dirty
if (!modelFieldsChanged) {
return
}
changed.push(value)
})
if (!changed.length && !visibilityChanged) {
return
}
let saveOperation = of([])
if (changed.length) {
this.savedViewService.patchMany(changed).subscribe({
saveOperation = saveOperation.pipe(
switchMap(() => this.savedViewService.patchMany(changed))
)
}
if (visibilityChanged) {
saveOperation = saveOperation.pipe(
switchMap(() =>
this.settings.updateSavedViewsVisibility(
dashboardVisibleIds,
sidebarVisibleIds
)
)
)
}
saveOperation.subscribe({
next: () => {
this.toastService.showInfo($localize`Views saved successfully.`)
this.savedViewService.clearCache()
this.reloadViews()
},
error: (error) => {
this.toastService.showError($localize`Error while saving views.`, error)
},
})
}
public canEditSavedView(view: SavedView): boolean {
return this.permissionsService.currentUserHasObjectPermissions(
PermissionAction.Change,
view
)
}
public canDeleteSavedView(view: SavedView): boolean {
return this.permissionsService.currentUserOwnsObject(view)
}
public editPermissions(savedView: SavedView): void {
const modal = this.modalService.open(PermissionsDialogComponent, {
backdrop: 'static',
})
const dialog = modal.componentInstance as PermissionsDialogComponent
dialog.object = savedView
dialog.note = $localize`Note: Sharing saved views does not share the underlying documents.`
modal.componentInstance.confirmClicked.subscribe(({ permissions }) => {
modal.componentInstance.buttonsEnabled = false
const view = {
id: savedView.id,
owner: permissions.owner,
}
view['set_permissions'] = permissions.set_permissions
this.savedViewService.patch(view as SavedView).subscribe({
next: () => {
this.toastService.showInfo($localize`Views saved successfully.`)
this.store.next(this.savedViewsForm.value)
this.toastService.showInfo($localize`Permissions updated`)
modal.close()
this.reloadViews()
},
error: (error) => {
this.toastService.showError(
$localize`Error while saving views.`,
$localize`Error updating permissions`,
error
)
},
})
}
})
}
}

View File

@@ -128,15 +128,15 @@ export interface Document extends ObjectWithPermissions {
checksum?: string
// UTC
created?: Date
created?: string // ISO string
modified?: Date
modified?: string // ISO string
added?: Date
added?: string // ISO string
mime_type?: string
deleted_at?: Date
deleted_at?: string // ISO string
original_file_name?: string

View File

@@ -62,6 +62,10 @@ export const SETTINGS_KEYS = {
'general-settings:update-checking:backend-setting',
SAVED_VIEWS_WARN_ON_UNSAVED_CHANGE:
'general-settings:saved-views:warn-on-unsaved-change',
DASHBOARD_VIEWS_VISIBLE_IDS:
'general-settings:saved-views:dashboard-views-visible-ids',
SIDEBAR_VIEWS_VISIBLE_IDS:
'general-settings:saved-views:sidebar-views-visible-ids',
DASHBOARD_VIEWS_SORT_ORDER:
'general-settings:saved-views:dashboard-views-sort-order',
SIDEBAR_VIEWS_SORT_ORDER:
@@ -248,6 +252,16 @@ export const SETTINGS: UiSetting[] = [
type: 'array',
default: [],
},
{
key: SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS,
type: 'array',
default: [],
},
{
key: SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS,
type: 'array',
default: [],
},
{
key: SETTINGS_KEYS.DASHBOARD_VIEWS_SORT_ORDER,
type: 'array',

View File

@@ -0,0 +1,7 @@
export interface WebsocketDocumentUpdatedMessage {
document_id: number
modified: string
owner_id?: number
users_can_view?: number[]
groups_can_view?: number[]
}

View File

@@ -57,6 +57,11 @@ describe(`Additional service tests for SavedViewService`, () => {
let settingsService
it('should retrieve saved views and sort them', () => {
jest.spyOn(settingsService, 'get').mockImplementation((key) => {
if (key === SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS) return [1, 2, 3]
if (key === SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS) return [1, 2, 3]
return []
})
service.reload()
const req = httpTestingController.expectOne(
`${environment.apiBaseUrl}${endpoint}/?page=1&page_size=100000`
@@ -93,7 +98,9 @@ describe(`Additional service tests for SavedViewService`, () => {
it('should sort dashboard views', () => {
service['savedViews'] = saved_views
jest.spyOn(settingsService, 'get').mockImplementation((key) => {
if (key === SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS) return [1, 2, 3]
if (key === SETTINGS_KEYS.DASHBOARD_VIEWS_SORT_ORDER) return [3, 1, 2]
return []
})
expect(service.dashboardViews).toEqual([
saved_views[2],
@@ -102,10 +109,21 @@ describe(`Additional service tests for SavedViewService`, () => {
])
})
it('should use user-specific dashboard visibility when configured', () => {
service['savedViews'] = saved_views
jest.spyOn(settingsService, 'get').mockImplementation((key) => {
if (key === SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS) return [4, 2]
if (key === SETTINGS_KEYS.DASHBOARD_VIEWS_SORT_ORDER) return []
})
expect(service.dashboardViews).toEqual([saved_views[1], saved_views[3]])
})
it('should sort sidebar views', () => {
service['savedViews'] = saved_views
jest.spyOn(settingsService, 'get').mockImplementation((key) => {
if (key === SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS) return [1, 2, 3]
if (key === SETTINGS_KEYS.SIDEBAR_VIEWS_SORT_ORDER) return [3, 1, 2]
return []
})
expect(service.sidebarViews).toEqual([
saved_views[2],
@@ -114,6 +132,15 @@ describe(`Additional service tests for SavedViewService`, () => {
])
})
it('should use user-specific sidebar visibility when configured', () => {
service['savedViews'] = saved_views
jest.spyOn(settingsService, 'get').mockImplementation((key) => {
if (key === SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS) return [4, 2]
if (key === SETTINGS_KEYS.SIDEBAR_VIEWS_SORT_ORDER) return []
})
expect(service.sidebarViews).toEqual([saved_views[1], saved_views[3]])
})
it('should treat empty display_fields as null', () => {
subscription = service
.patch({

View File

@@ -36,7 +36,9 @@ export class SavedViewService extends AbstractPaperlessService<SavedView> {
return super.list(page, pageSize, sortField, sortReverse, extraParams).pipe(
tap({
next: (r) => {
this.savedViews = r.results
const views = r.results.map((view) => this.withUserVisibility(view))
this.savedViews = views
r.results = views
this._loading = false
this.settingsService.dashboardIsEmpty =
this.dashboardViews.length === 0
@@ -65,8 +67,35 @@ export class SavedViewService extends AbstractPaperlessService<SavedView> {
return this.savedViews
}
private getVisibleViewIds(setting: string): number[] {
const configured = this.settingsService.get(setting)
return Array.isArray(configured) ? configured : []
}
private withUserVisibility(view: SavedView): SavedView {
return {
...view,
show_on_dashboard: this.isDashboardVisible(view),
show_in_sidebar: this.isSidebarVisible(view),
}
}
private isDashboardVisible(view: SavedView): boolean {
const visibleIds = this.getVisibleViewIds(
SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS
)
return visibleIds.includes(view.id)
}
private isSidebarVisible(view: SavedView): boolean {
const visibleIds = this.getVisibleViewIds(
SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS
)
return visibleIds.includes(view.id)
}
get sidebarViews(): SavedView[] {
const sidebarViews = this.savedViews.filter((v) => v.show_in_sidebar)
const sidebarViews = this.savedViews.filter((v) => this.isSidebarVisible(v))
const sorted: number[] = this.settingsService.get(
SETTINGS_KEYS.SIDEBAR_VIEWS_SORT_ORDER
@@ -81,7 +110,9 @@ export class SavedViewService extends AbstractPaperlessService<SavedView> {
}
get dashboardViews(): SavedView[] {
const dashboardViews = this.savedViews.filter((v) => v.show_on_dashboard)
const dashboardViews = this.savedViews.filter((v) =>
this.isDashboardVisible(v)
)
const sorted: number[] = this.settingsService.get(
SETTINGS_KEYS.DASHBOARD_VIEWS_SORT_ORDER

View File

@@ -320,7 +320,7 @@ describe('SettingsService', () => {
expect(req.request.method).toEqual('POST')
})
it('should update saved view sorting', () => {
it('should update saved view sorting and visibility', () => {
httpTestingController
.expectOne(`${environment.apiBaseUrl}ui_settings/`)
.flush(ui_settings)
@@ -341,6 +341,15 @@ describe('SettingsService', () => {
SETTINGS_KEYS.SIDEBAR_VIEWS_SORT_ORDER,
[1, 4]
)
settingsService.updateSavedViewsVisibility([1, 4], [4, 1])
expect(setSpy).toHaveBeenCalledWith(
SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS,
[1, 4]
)
expect(setSpy).toHaveBeenCalledWith(
SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS,
[4, 1]
)
httpTestingController
.expectOne(`${environment.apiBaseUrl}ui_settings/`)
.flush(ui_settings)

View File

@@ -699,4 +699,17 @@ export class SettingsService {
])
return this.storeSettings()
}
updateSavedViewsVisibility(
dashboardVisibleViewIds: number[],
sidebarVisibleViewIds: number[]
): Observable<any> {
this.set(SETTINGS_KEYS.DASHBOARD_VIEWS_VISIBLE_IDS, [
...new Set(dashboardVisibleViewIds),
])
this.set(SETTINGS_KEYS.SIDEBAR_VIEWS_VISIBLE_IDS, [
...new Set(sidebarVisibleViewIds),
])
return this.storeSettings()
}
}

View File

@@ -416,4 +416,42 @@ describe('ConsumerStatusService', () => {
websocketStatusService.disconnect()
expect(deleted).toBeTruthy()
})
it('should trigger updated subject on document updated', () => {
let updated = false
websocketStatusService.onDocumentUpdated().subscribe((data) => {
updated = true
expect(data.document_id).toEqual(12)
})
websocketStatusService.connect()
server.send({
type: WebsocketStatusType.DOCUMENT_UPDATED,
data: {
document_id: 12,
modified: '2026-02-17T00:00:00Z',
owner_id: 1,
},
})
websocketStatusService.disconnect()
expect(updated).toBeTruthy()
})
it('should ignore document updated events the user cannot view', () => {
let updated = false
websocketStatusService.onDocumentUpdated().subscribe(() => {
updated = true
})
websocketStatusService.handleDocumentUpdated({
document_id: 12,
modified: '2026-02-17T00:00:00Z',
owner_id: 2,
users_can_view: [],
groups_can_view: [],
})
expect(updated).toBeFalsy()
})
})

View File

@@ -2,6 +2,7 @@ import { Injectable, inject } from '@angular/core'
import { Subject } from 'rxjs'
import { environment } from 'src/environments/environment'
import { User } from '../data/user'
import { WebsocketDocumentUpdatedMessage } from '../data/websocket-document-updated-message'
import { WebsocketDocumentsDeletedMessage } from '../data/websocket-documents-deleted-message'
import { WebsocketProgressMessage } from '../data/websocket-progress-message'
import { SettingsService } from './settings.service'
@@ -9,6 +10,7 @@ import { SettingsService } from './settings.service'
export enum WebsocketStatusType {
STATUS_UPDATE = 'status_update',
DOCUMENTS_DELETED = 'documents_deleted',
DOCUMENT_UPDATED = 'document_updated',
}
// see ProgressStatusOptions in src/documents/plugins/helpers.py
@@ -100,17 +102,20 @@ export enum UploadState {
providedIn: 'root',
})
export class WebsocketStatusService {
private settingsService = inject(SettingsService)
private readonly settingsService = inject(SettingsService)
private statusWebSocket: WebSocket
private consumerStatus: FileStatus[] = []
private documentDetectedSubject = new Subject<FileStatus>()
private documentConsumptionFinishedSubject = new Subject<FileStatus>()
private documentConsumptionFailedSubject = new Subject<FileStatus>()
private documentDeletedSubject = new Subject<boolean>()
private connectionStatusSubject = new Subject<boolean>()
private readonly documentDetectedSubject = new Subject<FileStatus>()
private readonly documentConsumptionFinishedSubject =
new Subject<FileStatus>()
private readonly documentConsumptionFailedSubject = new Subject<FileStatus>()
private readonly documentDeletedSubject = new Subject<boolean>()
private readonly documentUpdatedSubject =
new Subject<WebsocketDocumentUpdatedMessage>()
private readonly connectionStatusSubject = new Subject<boolean>()
private get(taskId: string, filename?: string) {
let status =
@@ -176,7 +181,10 @@ export class WebsocketStatusService {
data: messageData,
}: {
type: WebsocketStatusType
data: WebsocketProgressMessage | WebsocketDocumentsDeletedMessage
data:
| WebsocketProgressMessage
| WebsocketDocumentsDeletedMessage
| WebsocketDocumentUpdatedMessage
} = JSON.parse(ev.data)
switch (type) {
@@ -184,6 +192,12 @@ export class WebsocketStatusService {
this.documentDeletedSubject.next(true)
break
case WebsocketStatusType.DOCUMENT_UPDATED:
this.handleDocumentUpdated(
messageData as WebsocketDocumentUpdatedMessage
)
break
case WebsocketStatusType.STATUS_UPDATE:
this.handleProgressUpdate(messageData as WebsocketProgressMessage)
break
@@ -191,7 +205,11 @@ export class WebsocketStatusService {
}
}
private canViewMessage(messageData: WebsocketProgressMessage): boolean {
private canViewMessage(messageData: {
owner_id?: number
users_can_view?: number[]
groups_can_view?: number[]
}): boolean {
// see paperless.consumers.StatusConsumer._can_view
const user: User = this.settingsService.currentUser
return (
@@ -251,6 +269,15 @@ export class WebsocketStatusService {
}
}
handleDocumentUpdated(messageData: WebsocketDocumentUpdatedMessage) {
// fallback if backend didn't restrict message
if (!this.canViewMessage(messageData)) {
return
}
this.documentUpdatedSubject.next(messageData)
}
fail(status: FileStatus, message: string) {
status.message = message
status.phase = FileStatusPhase.FAILED
@@ -304,6 +331,10 @@ export class WebsocketStatusService {
return this.documentDeletedSubject
}
onDocumentUpdated() {
return this.documentUpdatedSubject
}
onConnectionStatus() {
return this.connectionStatusSubject.asObservable()
}

View File

@@ -3,10 +3,10 @@ const base_url = new URL(document.baseURI)
export const environment = {
production: true,
apiBaseUrl: document.baseURI + 'api/',
apiVersion: '9', // match src/paperless/settings.py
apiVersion: '10', // match src/paperless/settings.py
appTitle: 'Paperless-ngx',
tag: 'prod',
version: '2.20.8',
version: '2.20.10',
webSocketHost: window.location.host,
webSocketProtocol: window.location.protocol == 'https:' ? 'wss:' : 'ws:',
webSocketBaseUrl: base_url.pathname + 'ws/',

View File

@@ -15,6 +15,7 @@ class DocumentsConfig(AppConfig):
from documents.signals.handlers import add_to_index
from documents.signals.handlers import run_workflows_added
from documents.signals.handlers import run_workflows_updated
from documents.signals.handlers import send_websocket_document_updated
from documents.signals.handlers import set_correspondent
from documents.signals.handlers import set_document_type
from documents.signals.handlers import set_storage_path
@@ -29,6 +30,7 @@ class DocumentsConfig(AppConfig):
document_consumption_finished.connect(run_workflows_added)
document_consumption_finished.connect(add_or_update_document_in_llm_index)
document_updated.connect(run_workflows_updated)
document_updated.connect(send_websocket_document_updated)
import documents.schema # noqa: F401

View File

@@ -1,5 +1,5 @@
from datetime import UTC
from datetime import datetime
from datetime import timezone
from typing import Any
from django.conf import settings
@@ -139,7 +139,7 @@ def thumbnail_last_modified(request: Any, pk: int) -> datetime | None:
# No cache, get the timestamp and cache the datetime
last_modified = datetime.fromtimestamp(
doc.thumbnail_path.stat().st_mtime,
tz=timezone.utc,
tz=UTC,
)
cache.set(doc_key, last_modified, CACHE_50_MINUTES)
return last_modified

View File

@@ -2,7 +2,7 @@ import datetime
import hashlib
import os
import tempfile
from enum import Enum
from enum import StrEnum
from pathlib import Path
from typing import TYPE_CHECKING
from typing import Final
@@ -11,6 +11,7 @@ import magic
from django.conf import settings
from django.contrib.auth.models import User
from django.db import transaction
from django.db.models import Max
from django.db.models import Q
from django.utils import timezone
from filelock import FileLock
@@ -20,6 +21,7 @@ from documents.classifier import load_classifier
from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from documents.file_handling import create_source_path_directory
from documents.file_handling import generate_filename
from documents.file_handling import generate_unique_filename
from documents.loggers import LoggingMixin
from documents.models import Correspondent
@@ -43,6 +45,7 @@ from documents.plugins.helpers import ProgressManager
from documents.plugins.helpers import ProgressStatusOptions
from documents.signals import document_consumption_finished
from documents.signals import document_consumption_started
from documents.signals import document_updated
from documents.signals.handlers import run_workflows
from documents.templating.workflows import parse_w_workflow_placeholders
from documents.utils import copy_basic_file_stats
@@ -80,7 +83,7 @@ class ConsumerError(Exception):
pass
class ConsumerStatusShortMessage(str, Enum):
class ConsumerStatusShortMessage(StrEnum):
DOCUMENT_ALREADY_EXISTS = "document_already_exists"
DOCUMENT_ALREADY_EXISTS_IN_TRASH = "document_already_exists_in_trash"
ASN_ALREADY_EXISTS = "asn_already_exists"
@@ -122,22 +125,6 @@ class ConsumerPluginMixin:
self.filename = self.metadata.filename or self.input_doc.original_file.name
if input_doc.root_document_id:
self.log.debug(
f"Document root document id: {input_doc.root_document_id}",
)
root_document = Document.objects.get(pk=input_doc.root_document_id)
version_index = Document.objects.filter(root_document=root_document).count()
filename_path = Path(self.filename)
if filename_path.suffix:
self.filename = str(
filename_path.with_name(
f"{filename_path.stem}_v{version_index}{filename_path.suffix}",
),
)
else:
self.filename = f"{self.filename}_v{version_index}"
def _send_progress(
self,
current_progress: int,
@@ -183,7 +170,7 @@ class ConsumerPlugin(
):
logging_name = LOGGING_NAME
def _clone_root_into_version(
def _create_version_from_root(
self,
root_doc: Document,
*,
@@ -192,30 +179,38 @@ class ConsumerPlugin(
mime_type: str,
) -> Document:
self.log.debug("Saving record for updated version to database")
version_doc = Document.objects.get(pk=root_doc.pk)
setattr(version_doc, "pk", None)
version_doc.root_document = root_doc
root_doc_frozen = Document.objects.select_for_update().get(pk=root_doc.pk)
next_version_index = (
Document.global_objects.filter(
root_document_id=root_doc_frozen.pk,
).aggregate(
max_index=Max("version_index"),
)["max_index"]
or 0
)
file_for_checksum = (
self.unmodified_original
if self.unmodified_original is not None
else self.working_copy
)
version_doc.checksum = hashlib.md5(
file_for_checksum.read_bytes(),
).hexdigest()
version_doc.content = text or ""
version_doc.page_count = page_count
version_doc.mime_type = mime_type
version_doc.original_filename = self.filename
version_doc.storage_path = root_doc.storage_path
# Clear unique file path fields so they can be generated uniquely later
version_doc.filename = None
version_doc.archive_filename = None
version_doc.archive_checksum = None
version_doc = Document(
root_document=root_doc_frozen,
version_index=next_version_index + 1,
checksum=hashlib.md5(
file_for_checksum.read_bytes(),
).hexdigest(),
content=text or "",
page_count=page_count,
mime_type=mime_type,
original_filename=self.filename,
owner_id=root_doc_frozen.owner_id,
created=root_doc_frozen.created,
title=root_doc_frozen.title,
added=timezone.now(),
modified=timezone.now(),
)
if self.metadata.version_label is not None:
version_doc.version_label = self.metadata.version_label
version_doc.added = timezone.now()
version_doc.modified = timezone.now()
return version_doc
def run_pre_consume_script(self) -> None:
@@ -541,7 +536,7 @@ class ConsumerPlugin(
root_doc = Document.objects.get(
pk=self.input_doc.root_document_id,
)
original_document = self._clone_root_into_version(
original_document = self._create_version_from_root(
root_doc,
text=text,
page_count=page_count,
@@ -610,7 +605,19 @@ class ConsumerPlugin(
# After everything is in the database, copy the files into
# place. If this fails, we'll also rollback the transaction.
with FileLock(settings.MEDIA_LOCK):
document.filename = generate_unique_filename(document)
generated_filename = generate_unique_filename(document)
if (
len(str(generated_filename))
> Document.MAX_STORED_FILENAME_LENGTH
):
self.log.warning(
"Generated source filename exceeds db path limit, falling back to default naming",
)
generated_filename = generate_filename(
document,
use_format=False,
)
document.filename = generated_filename
create_source_path_directory(document.source_path)
self._write(
@@ -626,10 +633,23 @@ class ConsumerPlugin(
)
if archive_path and Path(archive_path).is_file():
document.archive_filename = generate_unique_filename(
generated_archive_filename = generate_unique_filename(
document,
archive_filename=True,
)
if (
len(str(generated_archive_filename))
> Document.MAX_STORED_FILENAME_LENGTH
):
self.log.warning(
"Generated archive filename exceeds db path limit, falling back to default naming",
)
generated_archive_filename = generate_filename(
document,
archive_filename=True,
use_format=False,
)
document.archive_filename = generated_archive_filename
create_source_path_directory(document.archive_path)
self._write(
archive_path,
@@ -646,6 +666,12 @@ class ConsumerPlugin(
# This triggers things like file renaming
document.save()
if document.root_document_id:
document_updated.send(
sender=self.__class__,
document=document.root_document,
)
# Delete the file only if it was successfully consumed
self.log.debug(f"Deleting original file {self.input_doc.original_file}")
self.input_doc.original_file.unlink()

View File

@@ -127,23 +127,34 @@ def generate_filename(
*,
counter=0,
archive_filename=False,
use_format=True,
) -> Path:
# version docs use the root document for formatting, just with a suffix
context_doc = doc if doc.root_document_id is None else doc.root_document
version_suffix = (
f"_v{doc.version_index}"
if doc.root_document_id is not None and doc.version_index is not None
else ""
)
base_path: Path | None = None
# Determine the source of the format string
if doc.storage_path is not None:
filename_format = doc.storage_path.path
elif settings.FILENAME_FORMAT is not None:
# Maybe convert old to new style
filename_format = convert_format_str_to_template_format(
settings.FILENAME_FORMAT,
)
if use_format:
if context_doc.storage_path is not None:
filename_format = context_doc.storage_path.path
elif settings.FILENAME_FORMAT is not None:
# Maybe convert old to new style
filename_format = convert_format_str_to_template_format(
settings.FILENAME_FORMAT,
)
else:
filename_format = None
else:
filename_format = None
# If we have one, render it
if filename_format is not None:
rendered_path: str | None = format_filename(doc, filename_format)
rendered_path: str | None = format_filename(context_doc, filename_format)
if rendered_path:
base_path = Path(rendered_path)
@@ -157,7 +168,7 @@ def generate_filename(
base_filename = base_path.name
# Build the final filename with counter and filetype
final_filename = f"{base_filename}{counter_str}{filetype_str}"
final_filename = f"{base_filename}{version_suffix}{counter_str}{filetype_str}"
# If we have a directory component, include it
if str(directory) != ".":
@@ -166,7 +177,9 @@ def generate_filename(
full_path = Path(final_filename)
else:
# No template, use document ID
final_filename = f"{doc.pk:07}{counter_str}{filetype_str}"
final_filename = (
f"{context_doc.pk:07}{version_suffix}{counter_str}{filetype_str}"
)
full_path = Path(final_filename)
return full_path

View File

@@ -5,10 +5,10 @@ import math
import re
from collections import Counter
from contextlib import contextmanager
from datetime import UTC
from datetime import datetime
from datetime import time
from datetime import timedelta
from datetime import timezone
from shutil import rmtree
from time import sleep
from typing import TYPE_CHECKING
@@ -437,7 +437,7 @@ class ManualResults:
class LocalDateParser(English):
def reverse_timezone_offset(self, d):
return (d.replace(tzinfo=django_timezone.get_current_timezone())).astimezone(
timezone.utc,
UTC,
)
def date_from(self, *args, **kwargs):
@@ -641,8 +641,8 @@ def rewrite_natural_date_keywords(query_string: str) -> str:
end = datetime(local_now.year - 1, 12, 31, 23, 59, 59, tzinfo=tz)
# Convert to UTC and format
start_str = start.astimezone(timezone.utc).strftime("%Y%m%d%H%M%S")
end_str = end.astimezone(timezone.utc).strftime("%Y%m%d%H%M%S")
start_str = start.astimezone(UTC).strftime("%Y%m%d%H%M%S")
end_str = end.astimezone(UTC).strftime("%Y%m%d%H%M%S")
return f"{field}:[{start_str} TO {end_str}]"
return re.sub(pattern, repl, query_string, flags=re.IGNORECASE)

View File

@@ -0,0 +1,550 @@
"""
Base command class for Paperless-ngx management commands.
Provides automatic progress bar and multiprocessing support with minimal boilerplate.
"""
from __future__ import annotations
import logging
import os
from collections.abc import Callable
from collections.abc import Iterable
from collections.abc import Sized
from concurrent.futures import ProcessPoolExecutor
from concurrent.futures import as_completed
from contextlib import contextmanager
from dataclasses import dataclass
from typing import TYPE_CHECKING
from typing import Any
from typing import ClassVar
from typing import Generic
from typing import TypeVar
from django import db
from django.core.management import CommandError
from django.db.models import QuerySet
from django_rich.management import RichCommand
from rich import box
from rich.console import Console
from rich.console import Group
from rich.console import RenderableType
from rich.live import Live
from rich.progress import BarColumn
from rich.progress import MofNCompleteColumn
from rich.progress import Progress
from rich.progress import SpinnerColumn
from rich.progress import TextColumn
from rich.progress import TimeElapsedColumn
from rich.progress import TimeRemainingColumn
from rich.table import Table
from rich.text import Text
if TYPE_CHECKING:
from collections.abc import Generator
from collections.abc import Sequence
from django.core.management import CommandParser
T = TypeVar("T")
R = TypeVar("R")
@dataclass(slots=True, frozen=True)
class _BufferedRecord:
level: int
name: str
message: str
class BufferingLogHandler(logging.Handler):
"""Captures log records during a command run for deferred rendering.
Attach to a logger before a long operation and call ``render()``
afterwards to emit the buffered records via Rich, optionally filtered
by minimum level.
"""
def __init__(self) -> None:
super().__init__()
self._records: list[_BufferedRecord] = []
def emit(self, record: logging.LogRecord) -> None:
self._records.append(
_BufferedRecord(
level=record.levelno,
name=record.name,
message=self.format(record),
),
)
def render(
self,
console: Console,
*,
min_level: int = logging.DEBUG,
title: str = "Log Output",
) -> None:
records = [r for r in self._records if r.level >= min_level]
if not records:
return
table = Table(
title=title,
show_header=True,
header_style="bold",
show_lines=False,
box=box.SIMPLE,
)
table.add_column("Level", style="bold", width=8)
table.add_column("Logger", style="dim")
table.add_column("Message", no_wrap=False)
_level_styles: dict[int, str] = {
logging.DEBUG: "dim",
logging.INFO: "cyan",
logging.WARNING: "yellow",
logging.ERROR: "red",
logging.CRITICAL: "bold red",
}
for record in records:
style = _level_styles.get(record.level, "")
table.add_row(
Text(logging.getLevelName(record.level), style=style),
record.name,
record.message,
)
console.print(table)
def clear(self) -> None:
self._records.clear()
@dataclass(frozen=True, slots=True)
class ProcessResult(Generic[T, R]):
"""
Result of processing a single item in parallel.
Attributes:
item: The input item that was processed.
result: The return value from the processing function, or None if an error occurred.
error: The exception if processing failed, or None on success.
"""
item: T
result: R | None
error: BaseException | None
@property
def success(self) -> bool:
"""Return True if the item was processed successfully."""
return self.error is None
class PaperlessCommand(RichCommand):
"""
Base command class with automatic progress bar and multiprocessing support.
Features are opt-in via class attributes:
supports_progress_bar: Adds --no-progress-bar argument (default: True)
supports_multiprocessing: Adds --processes argument (default: False)
Example usage:
class Command(PaperlessCommand):
help = "Process all documents"
def handle(self, *args, **options):
documents = Document.objects.all()
for doc in self.track(documents, description="Processing..."):
process_document(doc)
class Command(PaperlessCommand):
help = "Regenerate thumbnails"
supports_multiprocessing = True
def handle(self, *args, **options):
ids = list(Document.objects.values_list("id", flat=True))
for result in self.process_parallel(process_doc, ids):
if result.error:
self.console.print(f"[red]Failed: {result.error}[/red]")
class Command(PaperlessCommand):
help = "Import documents with live stats"
def handle(self, *args, **options):
stats = ImportStats()
def render_stats() -> Table:
... # build Rich Table from stats
for item in self.track_with_stats(
items,
description="Importing...",
stats_renderer=render_stats,
):
result = import_item(item)
stats.imported += 1
"""
supports_progress_bar: ClassVar[bool] = True
supports_multiprocessing: ClassVar[bool] = False
# Instance attributes set by execute() before handle() runs
no_progress_bar: bool
process_count: int
def add_arguments(self, parser: CommandParser) -> None:
"""Add arguments based on supported features."""
super().add_arguments(parser)
if self.supports_progress_bar:
parser.add_argument(
"--no-progress-bar",
default=False,
action="store_true",
help="Disable the progress bar",
)
if self.supports_multiprocessing:
default_processes = max(1, (os.cpu_count() or 1) // 4)
parser.add_argument(
"--processes",
default=default_processes,
type=int,
help=f"Number of processes to use (default: {default_processes})",
)
def execute(self, *args: Any, **options: Any) -> str | None:
"""
Set up instance state before handle() is called.
This is called by Django's command infrastructure after argument parsing
but before handle(). We use it to set instance attributes from options.
"""
if self.supports_progress_bar:
self.no_progress_bar = options.get("no_progress_bar", False)
else:
self.no_progress_bar = True
if self.supports_multiprocessing:
self.process_count = options.get("processes", 1)
if self.process_count < 1:
raise CommandError("--processes must be at least 1")
else:
self.process_count = 1
return super().execute(*args, **options)
@contextmanager
def buffered_logging(
self,
*logger_names: str,
level: int = logging.DEBUG,
) -> Generator[BufferingLogHandler, None, None]:
"""Context manager that captures log output from named loggers.
Installs a ``BufferingLogHandler`` on each named logger for the
duration of the block, suppressing propagation to avoid interleaving
with the Rich live display. The handler is removed on exit regardless
of whether an exception occurred.
Usage::
with self.buffered_logging("paperless", "documents") as log_buf:
# ... run progress loop ...
if options["verbose"]:
log_buf.render(self.console)
"""
handler = BufferingLogHandler()
handler.setFormatter(logging.Formatter("%(message)s"))
loggers: list[logging.Logger] = []
original_propagate: dict[str, bool] = {}
for name in logger_names:
log = logging.getLogger(name)
log.addHandler(handler)
original_propagate[name] = log.propagate
log.propagate = False
loggers.append(log)
try:
yield handler
finally:
for log in loggers:
log.removeHandler(handler)
log.propagate = original_propagate[log.name]
@staticmethod
def _progress_columns() -> tuple[Any, ...]:
"""
Return the standard set of progress bar columns.
Extracted so both _create_progress (standalone) and track_with_stats
(inside Live) use identical column configuration without duplication.
"""
return (
SpinnerColumn(),
TextColumn("[progress.description]{task.description}"),
BarColumn(),
MofNCompleteColumn(),
TimeElapsedColumn(),
TimeRemainingColumn(),
)
def _create_progress(self, description: str) -> Progress:
"""
Create a standalone Progress instance with its own stderr Console.
Use this for track(). For track_with_stats(), Progress is created
directly inside a Live context instead.
Progress output is directed to stderr to match the convention that
progress bars are transient UI feedback, not command output. This
mirrors tqdm's default behavior and prevents progress bar rendering
from interfering with stdout-based assertions in tests or piped
command output.
Args:
description: Text to display alongside the progress bar.
Returns:
A Progress instance configured with appropriate columns.
"""
return Progress(
*self._progress_columns(),
console=Console(stderr=True),
transient=False,
)
def _get_iterable_length(self, iterable: Iterable[object]) -> int | None:
"""
Attempt to determine the length of an iterable without consuming it.
Tries .count() first (for Django querysets - executes SELECT COUNT(*)),
then falls back to len() for sequences.
Args:
iterable: The iterable to measure.
Returns:
The length if determinable, None otherwise.
"""
if isinstance(iterable, QuerySet):
return iterable.count()
if isinstance(iterable, Sized):
return len(iterable)
return None
def track(
self,
iterable: Iterable[T],
*,
description: str = "Processing...",
total: int | None = None,
) -> Generator[T, None, None]:
"""
Iterate over items with an optional progress bar.
Respects --no-progress-bar flag. When disabled, simply yields items
without any progress display.
Args:
iterable: The items to iterate over.
description: Text to display alongside the progress bar.
total: Total number of items. If None, attempts to determine
automatically via .count() (for querysets) or len().
Yields:
Items from the iterable.
Example:
for doc in self.track(documents, description="Renaming..."):
process(doc)
"""
if self.no_progress_bar:
yield from iterable
return
if total is None:
total = self._get_iterable_length(iterable)
with self._create_progress(description) as progress:
task_id = progress.add_task(description, total=total)
for item in iterable:
yield item
progress.advance(task_id)
def track_with_stats(
self,
iterable: Iterable[T],
*,
description: str = "Processing...",
stats_renderer: Callable[[], RenderableType],
total: int | None = None,
) -> Generator[T, None, None]:
"""
Iterate over items with a progress bar and a live-updating stats display.
The progress bar and stats renderable are combined in a single Live
context, so the stats panel re-renders in place below the progress bar
after each item is processed.
Respects --no-progress-bar flag. When disabled, yields items without
any display (stats are still updated by the caller's loop body, so
they will be accurate for any post-loop summary the caller prints).
Args:
iterable: The items to iterate over.
description: Text to display alongside the progress bar.
stats_renderer: Zero-argument callable that returns a Rich
renderable. Called after each item to refresh the display.
The caller typically closes over a mutable dataclass and
rebuilds a Table from it on each call.
total: Total number of items. If None, attempts to determine
automatically via .count() (for querysets) or len().
Yields:
Items from the iterable.
Example:
@dataclass
class Stats:
processed: int = 0
failed: int = 0
stats = Stats()
def render_stats() -> Table:
table = Table(box=None)
table.add_column("Processed")
table.add_column("Failed")
table.add_row(str(stats.processed), str(stats.failed))
return table
for item in self.track_with_stats(
items,
description="Importing...",
stats_renderer=render_stats,
):
try:
import_item(item)
stats.processed += 1
except Exception:
stats.failed += 1
"""
if self.no_progress_bar:
yield from iterable
return
if total is None:
total = self._get_iterable_length(iterable)
stderr_console = Console(stderr=True)
# Progress is created without its own console so Live controls rendering.
progress = Progress(*self._progress_columns())
task_id = progress.add_task(description, total=total)
with Live(
Group(progress, stats_renderer()),
console=stderr_console,
refresh_per_second=4,
) as live:
for item in iterable:
yield item
progress.advance(task_id)
live.update(Group(progress, stats_renderer()))
def process_parallel(
self,
fn: Callable[[T], R],
items: Sequence[T],
*,
description: str = "Processing...",
) -> Generator[ProcessResult[T, R], None, None]:
"""
Process items in parallel with progress tracking.
When --processes=1, runs sequentially in the main process without
spawning subprocesses. This is critical for testing, as multiprocessing
breaks fixtures, mocks, and database transactions.
When --processes > 1, uses ProcessPoolExecutor and automatically closes
database connections before spawning workers (required for PostgreSQL).
Args:
fn: Function to apply to each item. Must be picklable for parallel
execution (i.e., defined at module level, not a lambda or closure).
items: Sequence of items to process.
description: Text to display alongside the progress bar.
Yields:
ProcessResult for each item, containing the item, result, and any error.
Example:
def regenerate_thumbnail(doc_id: int) -> Path:
...
for result in self.process_parallel(regenerate_thumbnail, doc_ids):
if result.error:
self.console.print(f"[red]Failed {result.item}[/red]")
"""
total = len(items)
if self.process_count == 1:
# Sequential execution in main process - critical for testing, so we don't fork in fork, etc
yield from self._process_sequential(fn, items, description, total)
else:
# Parallel execution with ProcessPoolExecutor
yield from self._process_parallel(fn, items, description, total)
def _process_sequential(
self,
fn: Callable[[T], R],
items: Sequence[T],
description: str,
total: int,
) -> Generator[ProcessResult[T, R], None, None]:
"""Process items sequentially in the main process."""
for item in self.track(items, description=description, total=total):
try:
result = fn(item)
yield ProcessResult(item=item, result=result, error=None)
except Exception as e:
yield ProcessResult(item=item, result=None, error=e)
def _process_parallel(
self,
fn: Callable[[T], R],
items: Sequence[T],
description: str,
total: int,
) -> Generator[ProcessResult[T, R], None, None]:
"""Process items in parallel using ProcessPoolExecutor."""
# Close database connections before forking - required for PostgreSQL
db.connections.close_all()
with self._create_progress(description) as progress:
task_id = progress.add_task(description, total=total)
with ProcessPoolExecutor(max_workers=self.process_count) as executor:
# Submit all tasks and map futures back to items
future_to_item = {executor.submit(fn, item): item for item in items}
# Yield results as they complete
for future in as_completed(future_to_item):
item = future_to_item[future]
try:
result = future.result()
yield ProcessResult(item=item, result=result, error=None)
except Exception as e:
yield ProcessResult(item=item, result=None, error=e)
finally:
progress.advance(task_id)

View File

@@ -1,20 +1,15 @@
import logging
import multiprocessing
import tqdm
from django import db
from django.conf import settings
from django.core.management.base import BaseCommand
from documents.management.commands.mixins import MultiProcessMixin
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
from documents.tasks import update_document_content_maybe_archive_file
logger = logging.getLogger("paperless.management.archiver")
class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = (
"Using the current classification model, assigns correspondents, tags "
"and document types to all documents, effectively allowing you to "
@@ -22,7 +17,10 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
"modified) after their initial import."
)
supports_multiprocessing = True
def add_arguments(self, parser):
super().add_arguments(parser)
parser.add_argument(
"-f",
"--overwrite",
@@ -44,13 +42,8 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
"run on this specific document."
),
)
self.add_argument_progress_bar_mixin(parser)
self.add_argument_processes_mixin(parser)
def handle(self, *args, **options):
self.handle_processes_mixin(**options)
self.handle_progress_bar_mixin(**options)
settings.SCRATCH_DIR.mkdir(parents=True, exist_ok=True)
overwrite = options["overwrite"]
@@ -60,35 +53,21 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
else:
documents = Document.objects.all()
document_ids = list(
map(
lambda doc: doc.id,
filter(lambda d: overwrite or not d.has_archive_version, documents),
),
)
# Note to future self: this prevents django from reusing database
# connections between processes, which is bad and does not work
# with postgres.
db.connections.close_all()
document_ids = [
doc.id for doc in documents if overwrite or not doc.has_archive_version
]
try:
logging.getLogger().handlers[0].level = logging.ERROR
if self.process_count == 1:
for doc_id in document_ids:
update_document_content_maybe_archive_file(doc_id)
else: # pragma: no cover
with multiprocessing.Pool(self.process_count) as pool:
list(
tqdm.tqdm(
pool.imap_unordered(
update_document_content_maybe_archive_file,
document_ids,
),
total=len(document_ids),
disable=self.no_progress_bar,
),
for result in self.process_parallel(
update_document_content_maybe_archive_file,
document_ids,
description="Archiving...",
):
if result.error:
self.console.print(
f"[red]Failed document {result.item}: {result.error}[/red]",
)
except KeyboardInterrupt:
self.stdout.write(self.style.NOTICE("Aborting..."))
except KeyboardInterrupt: # pragma: no cover
self.console.print("[yellow]Aborting...[/yellow]")

View File

@@ -3,6 +3,8 @@ import json
import os
import shutil
import tempfile
from itertools import chain
from itertools import islice
from pathlib import Path
from typing import TYPE_CHECKING
@@ -19,6 +21,7 @@ from django.contrib.contenttypes.models import ContentType
from django.core import serializers
from django.core.management.base import BaseCommand
from django.core.management.base import CommandError
from django.core.serializers.json import DjangoJSONEncoder
from django.db import transaction
from django.utils import timezone
from filelock import FileLock
@@ -26,6 +29,8 @@ from guardian.models import GroupObjectPermission
from guardian.models import UserObjectPermission
if TYPE_CHECKING:
from collections.abc import Generator
from django.db.models import QuerySet
if settings.AUDIT_LOG_ENABLED:
@@ -60,6 +65,22 @@ from paperless_mail.models import MailAccount
from paperless_mail.models import MailRule
def serialize_queryset_batched(
queryset: "QuerySet",
*,
batch_size: int = 500,
) -> "Generator[list[dict], None, None]":
"""Yield batches of serialized records from a QuerySet.
Each batch is a list of dicts in Django's Python serialization format.
Uses QuerySet.iterator() to avoid loading the full queryset into memory,
and islice to collect chunk-sized batches serialized in a single call.
"""
iterator = queryset.iterator(chunk_size=batch_size)
while chunk := list(islice(iterator, batch_size)):
yield serializers.serialize("python", chunk)
class Command(CryptMixin, BaseCommand):
help = (
"Decrypt and rename all files in our collection into a given target "
@@ -186,6 +207,17 @@ class Command(CryptMixin, BaseCommand):
help="If provided, is used to encrypt sensitive data in the export",
)
parser.add_argument(
"--batch-size",
type=int,
default=500,
help=(
"Number of records to process per batch during serialization. "
"Lower values reduce peak memory usage; higher values improve "
"throughput. Default: 500."
),
)
def handle(self, *args, **options) -> None:
self.target = Path(options["target"]).resolve()
self.split_manifest: bool = options["split_manifest"]
@@ -200,6 +232,7 @@ class Command(CryptMixin, BaseCommand):
self.data_only: bool = options["data_only"]
self.no_progress_bar: bool = options["no_progress_bar"]
self.passphrase: str | None = options.get("passphrase")
self.batch_size: int = options["batch_size"]
self.files_in_export_dir: set[Path] = set()
self.exported_files: set[str] = set()
@@ -294,8 +327,13 @@ class Command(CryptMixin, BaseCommand):
# Build an overall manifest
for key, object_query in manifest_key_to_object_query.items():
manifest_dict[key] = json.loads(
serializers.serialize("json", object_query),
manifest_dict[key] = list(
chain.from_iterable(
serialize_queryset_batched(
object_query,
batch_size=self.batch_size,
),
),
)
self.encrypt_secret_fields(manifest_dict)
@@ -512,14 +550,24 @@ class Command(CryptMixin, BaseCommand):
self.files_in_export_dir.remove(target)
if self.compare_json:
target_checksum = hashlib.md5(target.read_bytes()).hexdigest()
src_str = json.dumps(content, indent=2, ensure_ascii=False)
src_str = json.dumps(
content,
cls=DjangoJSONEncoder,
indent=2,
ensure_ascii=False,
)
src_checksum = hashlib.md5(src_str.encode("utf-8")).hexdigest()
if src_checksum == target_checksum:
perform_write = False
if perform_write:
target.write_text(
json.dumps(content, indent=2, ensure_ascii=False),
json.dumps(
content,
cls=DjangoJSONEncoder,
indent=2,
ensure_ascii=False,
),
encoding="utf-8",
)

View File

@@ -1,24 +1,20 @@
import dataclasses
import multiprocessing
from typing import Final
import rapidfuzz
import tqdm
from django.core.management import BaseCommand
from django.core.management import CommandError
from documents.management.commands.mixins import MultiProcessMixin
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
@dataclasses.dataclass(frozen=True)
@dataclasses.dataclass(frozen=True, slots=True)
class _WorkPackage:
first_doc: Document
second_doc: Document
@dataclasses.dataclass(frozen=True)
@dataclasses.dataclass(frozen=True, slots=True)
class _WorkResult:
doc_one_pk: int
doc_two_pk: int
@@ -31,22 +27,23 @@ class _WorkResult:
def _process_and_match(work: _WorkPackage) -> _WorkResult:
"""
Does basic processing of document content, gets the basic ratio
and returns the result package
and returns the result package.
"""
# Normalize the string some, lower case, whitespace, etc
first_string = rapidfuzz.utils.default_process(work.first_doc.content)
second_string = rapidfuzz.utils.default_process(work.second_doc.content)
# Basic matching ratio
match = rapidfuzz.fuzz.ratio(first_string, second_string)
return _WorkResult(work.first_doc.pk, work.second_doc.pk, match)
class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = "Searches for documents where the content almost matches"
supports_multiprocessing = True
def add_arguments(self, parser):
super().add_arguments(parser)
parser.add_argument(
"--ratio",
default=85.0,
@@ -59,16 +56,11 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
action="store_true",
help="If set, one document of matches above the ratio WILL BE DELETED",
)
self.add_argument_progress_bar_mixin(parser)
self.add_argument_processes_mixin(parser)
def handle(self, *args, **options):
RATIO_MIN: Final[float] = 0.0
RATIO_MAX: Final[float] = 100.0
self.handle_processes_mixin(**options)
self.handle_progress_bar_mixin(**options)
if options["delete"]:
self.stdout.write(
self.style.WARNING(
@@ -80,66 +72,58 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
checked_pairs: set[tuple[int, int]] = set()
work_pkgs: list[_WorkPackage] = []
# Ratio is a float from 0.0 to 100.0
if opt_ratio < RATIO_MIN or opt_ratio > RATIO_MAX:
raise CommandError("The ratio must be between 0 and 100")
all_docs = Document.objects.all().order_by("id")
# Build work packages for processing
for first_doc in all_docs:
for second_doc in all_docs:
# doc to doc is obviously not useful
if first_doc.pk == second_doc.pk:
continue
# Skip empty documents (e.g. password-protected)
if first_doc.content.strip() == "" or second_doc.content.strip() == "":
continue
# Skip matching which have already been matched together
# doc 1 to doc 2 is the same as doc 2 to doc 1
doc_1_to_doc_2 = (first_doc.pk, second_doc.pk)
doc_2_to_doc_1 = doc_1_to_doc_2[::-1]
if doc_1_to_doc_2 in checked_pairs or doc_2_to_doc_1 in checked_pairs:
continue
checked_pairs.update([doc_1_to_doc_2, doc_2_to_doc_1])
# Actually something useful to work on now
work_pkgs.append(_WorkPackage(first_doc, second_doc))
# Don't spin up a pool of 1 process
results: list[_WorkResult] = []
if self.process_count == 1:
results = []
for work in tqdm.tqdm(work_pkgs, disable=self.no_progress_bar):
for work in self.track(work_pkgs, description="Matching..."):
results.append(_process_and_match(work))
else: # pragma: no cover
with multiprocessing.Pool(processes=self.process_count) as pool:
results = list(
tqdm.tqdm(
pool.imap_unordered(_process_and_match, work_pkgs),
total=len(work_pkgs),
disable=self.no_progress_bar,
),
)
for proc_result in self.process_parallel(
_process_and_match,
work_pkgs,
description="Matching...",
):
if proc_result.error:
self.console.print(
f"[red]Failed: {proc_result.error}[/red]",
)
elif proc_result.result is not None:
results.append(proc_result.result)
# Check results
messages = []
maybe_delete_ids = []
for result in sorted(results):
if result.ratio >= opt_ratio:
messages: list[str] = []
maybe_delete_ids: list[int] = []
for match_result in sorted(results):
if match_result.ratio >= opt_ratio:
messages.append(
self.style.NOTICE(
f"Document {result.doc_one_pk} fuzzy match"
f" to {result.doc_two_pk} (confidence {result.ratio:.3f})\n",
f"Document {match_result.doc_one_pk} fuzzy match"
f" to {match_result.doc_two_pk}"
f" (confidence {match_result.ratio:.3f})\n",
),
)
maybe_delete_ids.append(result.doc_two_pk)
maybe_delete_ids.append(match_result.doc_two_pk)
if len(messages) == 0:
messages.append(
self.style.SUCCESS("No matches found\n"),
)
self.stdout.writelines(
messages,
)
messages.append(self.style.SUCCESS("No matches found\n"))
self.stdout.writelines(messages)
if options["delete"]:
self.stdout.write(
self.style.NOTICE(

View File

@@ -1,22 +1,25 @@
from django.core.management import BaseCommand
from django.db import transaction
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.tasks import index_optimize
from documents.tasks import index_reindex
class Command(ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = "Manages the document index."
def add_arguments(self, parser):
super().add_arguments(parser)
parser.add_argument("command", choices=["reindex", "optimize"])
self.add_argument_progress_bar_mixin(parser)
def handle(self, *args, **options):
self.handle_progress_bar_mixin(**options)
with transaction.atomic():
if options["command"] == "reindex":
index_reindex(progress_bar_disable=self.no_progress_bar)
index_reindex(
iter_wrapper=lambda docs: self.track(
docs,
description="Indexing documents...",
),
)
elif options["command"] == "optimize":
index_optimize()

View File

@@ -1,22 +1,22 @@
from django.core.management import BaseCommand
from django.db import transaction
from typing import Any
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.tasks import llmindex_index
class Command(ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = "Manages the LLM-based vector index for Paperless."
def add_arguments(self, parser):
def add_arguments(self, parser: Any) -> None:
super().add_arguments(parser)
parser.add_argument("command", choices=["rebuild", "update"])
self.add_argument_progress_bar_mixin(parser)
def handle(self, *args, **options):
self.handle_progress_bar_mixin(**options)
with transaction.atomic():
llmindex_index(
progress_bar_disable=self.no_progress_bar,
rebuild=options["command"] == "rebuild",
scheduled=False,
)
def handle(self, *args: Any, **options: Any) -> None:
llmindex_index(
rebuild=options["command"] == "rebuild",
scheduled=False,
iter_wrapper=lambda docs: self.track(
docs,
description="Indexing documents...",
),
)

View File

@@ -1,25 +1,12 @@
import logging
import tqdm
from django.core.management.base import BaseCommand
from django.db.models.signals import post_save
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
class Command(ProgressBarMixin, BaseCommand):
help = "This will rename all documents to match the latest filename format."
def add_arguments(self, parser):
self.add_argument_progress_bar_mixin(parser)
class Command(PaperlessCommand):
help = "Rename all documents"
def handle(self, *args, **options):
self.handle_progress_bar_mixin(**options)
logging.getLogger().handlers[0].level = logging.ERROR
for document in tqdm.tqdm(
Document.objects.all(),
disable=self.no_progress_bar,
):
for document in self.track(Document.objects.all(), description="Renaming..."):
post_save.send(Document, instance=document, created=False)

View File

@@ -1,20 +1,178 @@
import logging
from __future__ import annotations
import tqdm
from django.core.management.base import BaseCommand
import logging
from dataclasses import dataclass
from dataclasses import field
from typing import TYPE_CHECKING
from rich.table import Table
from rich.text import Text
from documents.classifier import load_classifier
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
from documents.signals.handlers import set_correspondent
from documents.signals.handlers import set_document_type
from documents.signals.handlers import set_storage_path
from documents.signals.handlers import set_tags
if TYPE_CHECKING:
from rich.console import RenderableType
from documents.models import Correspondent
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
logger = logging.getLogger("paperless.management.retagger")
class Command(ProgressBarMixin, BaseCommand):
@dataclass(slots=True)
class RetaggerStats:
"""Cumulative counters updated as the retagger processes documents.
Mutable by design -- fields are incremented in the processing loop.
slots=True reduces per-instance memory overhead and speeds attribute access.
"""
correspondents: int = 0
document_types: int = 0
tags_added: int = 0
tags_removed: int = 0
storage_paths: int = 0
documents_processed: int = 0
@dataclass(slots=True)
class DocumentSuggestion:
"""Buffered classifier suggestions for a single document (suggest mode only).
Mutable by design -- fields are assigned incrementally as each setter runs.
"""
document: Document
correspondent: Correspondent | None = None
document_type: DocumentType | None = None
tags_to_add: frozenset[Tag] = field(default_factory=frozenset)
tags_to_remove: frozenset[Tag] = field(default_factory=frozenset)
storage_path: StoragePath | None = None
@property
def has_suggestions(self) -> bool:
return bool(
self.correspondent is not None
or self.document_type is not None
or self.tags_to_add
or self.tags_to_remove
or self.storage_path is not None,
)
def _build_stats_table(stats: RetaggerStats, *, suggest: bool) -> Table:
"""
Build the live-updating stats table shown below the progress bar.
In suggest mode the labels read "would set / would add" to make clear
that nothing has been written to the database.
"""
table = Table(box=None, padding=(0, 2), show_header=True, header_style="bold")
table.add_column("Documents")
table.add_column("Correspondents")
table.add_column("Doc Types")
table.add_column("Tags (+)")
table.add_column("Tags (-)")
table.add_column("Storage Paths")
verb = "would set" if suggest else "set"
table.add_row(
str(stats.documents_processed),
f"{stats.correspondents} {verb}",
f"{stats.document_types} {verb}",
f"+{stats.tags_added}",
f"-{stats.tags_removed}",
f"{stats.storage_paths} {verb}",
)
return table
def _build_suggestion_table(
suggestions: list[DocumentSuggestion],
base_url: str | None,
) -> Table:
"""
Build the final suggestion table printed after the progress bar completes.
Only documents with at least one suggestion are included.
"""
table = Table(
title="Suggested Changes",
show_header=True,
header_style="bold cyan",
show_lines=True,
)
table.add_column("Document", style="bold", no_wrap=False, min_width=20)
table.add_column("Correspondent")
table.add_column("Doc Type")
table.add_column("Tags")
table.add_column("Storage Path")
for suggestion in suggestions:
if not suggestion.has_suggestions:
continue
doc = suggestion.document
if base_url:
doc_cell = Text()
doc_cell.append(str(doc))
doc_cell.append(f"\n{base_url}/documents/{doc.pk}", style="dim")
else:
doc_cell = Text(f"{doc} [{doc.pk}]")
tag_parts: list[str] = []
for tag in sorted(suggestion.tags_to_add, key=lambda t: t.name):
tag_parts.append(f"[green]+{tag.name}[/green]")
for tag in sorted(suggestion.tags_to_remove, key=lambda t: t.name):
tag_parts.append(f"[red]-{tag.name}[/red]")
tag_cell = Text.from_markup(", ".join(tag_parts)) if tag_parts else Text("-")
table.add_row(
doc_cell,
str(suggestion.correspondent) if suggestion.correspondent else "-",
str(suggestion.document_type) if suggestion.document_type else "-",
tag_cell,
str(suggestion.storage_path) if suggestion.storage_path else "-",
)
return table
def _build_summary_table(stats: RetaggerStats) -> Table:
"""Build the final applied-changes summary table."""
table = Table(
title="Retagger Summary",
show_header=True,
header_style="bold cyan",
)
table.add_column("Metric", style="bold")
table.add_column("Count", justify="right")
table.add_row("Documents processed", str(stats.documents_processed))
table.add_row("Correspondents set", str(stats.correspondents))
table.add_row("Document types set", str(stats.document_types))
table.add_row("Tags added", str(stats.tags_added))
table.add_row("Tags removed", str(stats.tags_removed))
table.add_row("Storage paths set", str(stats.storage_paths))
return table
class Command(PaperlessCommand):
help = (
"Using the current classification model, assigns correspondents, tags "
"and document types to all documents, effectively allowing you to "
@@ -22,7 +180,8 @@ class Command(ProgressBarMixin, BaseCommand):
"modified) after their initial import."
)
def add_arguments(self, parser):
def add_arguments(self, parser) -> None:
super().add_arguments(parser)
parser.add_argument("-c", "--correspondent", default=False, action="store_true")
parser.add_argument("-T", "--tags", default=False, action="store_true")
parser.add_argument("-t", "--document_type", default=False, action="store_true")
@@ -33,9 +192,9 @@ class Command(ProgressBarMixin, BaseCommand):
default=False,
action="store_true",
help=(
"By default this command won't try to assign a correspondent "
"if more than one matches the document. Use this flag if "
"you'd rather it just pick the first one it finds."
"By default this command will not try to assign a correspondent "
"if more than one matches the document. Use this flag to pick "
"the first match instead."
),
)
parser.add_argument(
@@ -44,31 +203,44 @@ class Command(ProgressBarMixin, BaseCommand):
default=False,
action="store_true",
help=(
"If set, the document retagger will overwrite any previously "
"set correspondent, document and remove correspondents, types "
"and tags that do not match anymore due to changed rules."
"Overwrite any previously set correspondent, document type, and "
"remove tags that no longer match due to changed rules."
),
)
self.add_argument_progress_bar_mixin(parser)
parser.add_argument(
"--suggest",
default=False,
action="store_true",
help="Return the suggestion, don't change anything.",
help="Show what would be changed without applying anything.",
)
parser.add_argument(
"--base-url",
help="The base URL to use to build the link to the documents.",
help="Base URL used to build document links in suggest output.",
)
parser.add_argument(
"--id-range",
help="A range of document ids on which the retagging should be applied.",
help="Restrict retagging to documents within this ID range (inclusive).",
nargs=2,
type=int,
)
def handle(self, *args, **options):
self.handle_progress_bar_mixin(**options)
def handle(self, *args, **options) -> None:
suggest: bool = options["suggest"]
overwrite: bool = options["overwrite"]
use_first: bool = options["use_first"]
base_url: str | None = options["base_url"]
do_correspondent: bool = options["correspondent"]
do_document_type: bool = options["document_type"]
do_tags: bool = options["tags"]
do_storage_path: bool = options["storage_path"]
if not any([do_correspondent, do_document_type, do_tags, do_storage_path]):
self.console.print(
"[yellow]No classifier targets specified. "
"Use -c, -T, -t, or -s to select what to retag.[/yellow]",
)
return
if options["inbox_only"]:
queryset = Document.objects.filter(tags__is_inbox_tag=True)
@@ -76,61 +248,95 @@ class Command(ProgressBarMixin, BaseCommand):
queryset = Document.objects.all()
if options["id_range"]:
queryset = queryset.filter(
id__range=(options["id_range"][0], options["id_range"][1]),
)
lo, hi = options["id_range"]
queryset = queryset.filter(id__range=(lo, hi))
documents = queryset.distinct()
classifier = load_classifier()
for document in tqdm.tqdm(documents, disable=self.no_progress_bar):
if options["correspondent"]:
set_correspondent(
sender=None,
document=document,
classifier=classifier,
replace=options["overwrite"],
use_first=options["use_first"],
suggest=options["suggest"],
base_url=options["base_url"],
stdout=self.stdout,
style_func=self.style,
)
stats = RetaggerStats()
suggestions: list[DocumentSuggestion] = []
if options["document_type"]:
set_document_type(
sender=None,
document=document,
classifier=classifier,
replace=options["overwrite"],
use_first=options["use_first"],
suggest=options["suggest"],
base_url=options["base_url"],
stdout=self.stdout,
style_func=self.style,
)
def render_stats() -> RenderableType:
return _build_stats_table(stats, suggest=suggest)
if options["tags"]:
set_tags(
sender=None,
document=document,
classifier=classifier,
replace=options["overwrite"],
suggest=options["suggest"],
base_url=options["base_url"],
stdout=self.stdout,
style_func=self.style,
)
if options["storage_path"]:
set_storage_path(
sender=None,
document=document,
classifier=classifier,
replace=options["overwrite"],
use_first=options["use_first"],
suggest=options["suggest"],
base_url=options["base_url"],
stdout=self.stdout,
style_func=self.style,
)
with self.buffered_logging(
"paperless",
"paperless.handlers",
"documents",
) as log_buf:
for document in self.track_with_stats(
documents,
description="Retagging...",
stats_renderer=render_stats,
):
suggestion = DocumentSuggestion(document=document)
if do_correspondent:
correspondent = set_correspondent(
None,
document,
classifier=classifier,
replace=overwrite,
use_first=use_first,
dry_run=suggest,
)
if correspondent is not None:
stats.correspondents += 1
suggestion.correspondent = correspondent
if do_document_type:
document_type = set_document_type(
None,
document,
classifier=classifier,
replace=overwrite,
use_first=use_first,
dry_run=suggest,
)
if document_type is not None:
stats.document_types += 1
suggestion.document_type = document_type
if do_tags:
tags_to_add, tags_to_remove = set_tags(
None,
document,
classifier=classifier,
replace=overwrite,
dry_run=suggest,
)
stats.tags_added += len(tags_to_add)
stats.tags_removed += len(tags_to_remove)
suggestion.tags_to_add = frozenset(tags_to_add)
suggestion.tags_to_remove = frozenset(tags_to_remove)
if do_storage_path:
storage_path = set_storage_path(
None,
document,
classifier=classifier,
replace=overwrite,
use_first=use_first,
dry_run=suggest,
)
if storage_path is not None:
stats.storage_paths += 1
suggestion.storage_path = storage_path
stats.documents_processed += 1
if suggest:
suggestions.append(suggestion)
# Post-loop output
if suggest:
visible = [s for s in suggestions if s.has_suggestions]
if visible:
self.console.print(_build_suggestion_table(visible, base_url))
else:
self.console.print("[green]No changes suggested.[/green]")
else:
self.console.print(_build_summary_table(stats))
log_buf.render(self.console, min_level=logging.INFO, title="Retagger Log")

View File

@@ -1,17 +1,117 @@
from django.core.management.base import BaseCommand
"""Management command to check the document archive for issues."""
from documents.management.commands.mixins import ProgressBarMixin
from __future__ import annotations
import logging
from typing import Any
from rich.panel import Panel
from rich.table import Table
from rich.text import Text
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
from documents.sanity_checker import SanityCheckMessages
from documents.sanity_checker import check_sanity
_LEVEL_STYLE: dict[int, tuple[str, str]] = {
logging.ERROR: ("bold red", "ERROR"),
logging.WARNING: ("yellow", "WARN"),
logging.INFO: ("dim", "INFO"),
}
class Command(ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = "This command checks your document archive for issues."
def add_arguments(self, parser):
self.add_argument_progress_bar_mixin(parser)
def _render_results(self, messages: SanityCheckMessages) -> None:
"""Render sanity check results as a Rich table."""
def handle(self, *args, **options):
self.handle_progress_bar_mixin(**options)
messages = check_sanity(progress=self.use_progress_bar, scheduled=False)
if (
not messages.has_error
and not messages.has_warning
and not messages.has_info
):
self.console.print(
Panel(
"[green]No issues detected.[/green]",
title="Sanity Check",
border_style="green",
),
)
return
messages.log_messages()
# Build a lookup for document titles
doc_pks = [pk for pk in messages.document_pks() if pk is not None]
titles: dict[int, str] = {}
if doc_pks:
titles = dict(
Document.global_objects.filter(pk__in=doc_pks)
.only("pk", "title")
.values_list("pk", "title"),
)
table = Table(
title="Sanity Check Results",
show_lines=True,
title_style="bold",
)
table.add_column("Level", width=7, no_wrap=True)
table.add_column("Document", min_width=20)
table.add_column("Issue", ratio=1)
for doc_pk, doc_messages in messages.iter_messages():
if doc_pk is not None:
title = titles.get(doc_pk, "Unknown")
doc_label = f"#{doc_pk} {title}"
else:
doc_label = "(global)"
for msg in doc_messages:
style, label = _LEVEL_STYLE.get(
msg["level"],
("dim", "INFO"),
)
table.add_row(
Text(label, style=style),
Text(doc_label),
Text(str(msg["message"])),
)
self.console.print(table)
parts: list[str] = []
if messages.document_error_count:
parts.append(
f"{messages.document_error_count} document(s) with [bold red]errors[/bold red]",
)
if messages.document_warning_count:
parts.append(
f"{messages.document_warning_count} document(s) with [yellow]warnings[/yellow]",
)
if messages.document_info_count:
parts.append(f"{messages.document_info_count} document(s) with infos")
if messages.global_warning_count:
parts.append(
f"{messages.global_warning_count} global [yellow]warning(s)[/yellow]",
)
if parts:
if len(parts) > 1:
summary = ", ".join(parts[:-1]) + " and " + parts[-1]
else:
summary = parts[0]
self.console.print(f"\nFound {summary}.")
else:
self.console.print("\nNo issues found.")
def handle(self, *args: Any, **options: Any) -> None:
messages = check_sanity(
scheduled=False,
iter_wrapper=lambda docs: self.track(
docs,
description="Checking documents...",
),
)
self._render_results(messages)

View File

@@ -1,43 +1,45 @@
import logging
import multiprocessing
import shutil
import tqdm
from django import db
from django.core.management.base import BaseCommand
from documents.management.commands.mixins import MultiProcessMixin
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
from documents.models import Document
from documents.parsers import get_parser_class_for_mime_type
logger = logging.getLogger("paperless.management.thumbnails")
def _process_document(doc_id) -> None:
def _process_document(doc_id: int) -> None:
document: Document = Document.objects.get(id=doc_id)
parser_class = get_parser_class_for_mime_type(document.mime_type)
if parser_class:
parser = parser_class(logging_group=None)
else:
print(f"{document} No parser for mime type {document.mime_type}") # noqa: T201
if parser_class is None:
logger.warning(
"%s: No parser for mime type %s",
document,
document.mime_type,
)
return
parser = parser_class(logging_group=None)
try:
thumb = parser.get_thumbnail(
document.source_path,
document.mime_type,
document.get_public_filename(),
)
shutil.move(thumb, document.thumbnail_path)
finally:
parser.cleanup()
class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
class Command(PaperlessCommand):
help = "This will regenerate the thumbnails for all documents."
supports_multiprocessing = True
def add_arguments(self, parser) -> None:
super().add_arguments(parser)
parser.add_argument(
"-d",
"--document",
@@ -49,36 +51,23 @@ class Command(MultiProcessMixin, ProgressBarMixin, BaseCommand):
"run on this specific document."
),
)
self.add_argument_progress_bar_mixin(parser)
self.add_argument_processes_mixin(parser)
def handle(self, *args, **options):
logging.getLogger().handlers[0].level = logging.ERROR
self.handle_processes_mixin(**options)
self.handle_progress_bar_mixin(**options)
if options["document"]:
documents = Document.objects.filter(pk=options["document"])
else:
documents = Document.objects.all()
ids = [doc.id for doc in documents]
ids = list(documents.values_list("id", flat=True))
# Note to future self: this prevents django from reusing database
# connections between processes, which is bad and does not work
# with postgres.
db.connections.close_all()
if self.process_count == 1:
for doc_id in ids:
_process_document(doc_id)
else: # pragma: no cover
with multiprocessing.Pool(processes=self.process_count) as pool:
list(
tqdm.tqdm(
pool.imap_unordered(_process_document, ids),
total=len(ids),
disable=self.no_progress_bar,
),
for result in self.process_parallel(
_process_document,
ids,
description="Regenerating thumbnails...",
):
if result.error: # pragma: no cover
self.console.print(
f"[red]Failed document {result.item}: {result.error}[/red]",
)

View File

@@ -21,26 +21,6 @@ class CryptFields(TypedDict):
fields: list[str]
class MultiProcessMixin:
"""
Small class to handle adding an argument and validating it
for the use of multiple processes
"""
def add_argument_processes_mixin(self, parser: ArgumentParser) -> None:
parser.add_argument(
"--processes",
default=max(1, os.cpu_count() // 4),
type=int,
help="Number of processes to distribute work amongst",
)
def handle_processes_mixin(self, *args, **options) -> None:
self.process_count = options["processes"]
if self.process_count < 1:
raise CommandError("There must be at least 1 process")
class ProgressBarMixin:
"""
Many commands use a progress bar, which can be disabled

View File

@@ -1,27 +1,21 @@
from auditlog.models import LogEntry
from django.core.management.base import BaseCommand
from django.db import transaction
from tqdm import tqdm
from documents.management.commands.mixins import ProgressBarMixin
from documents.management.commands.base import PaperlessCommand
class Command(BaseCommand, ProgressBarMixin):
"""
Prune the audit logs of objects that no longer exist.
"""
class Command(PaperlessCommand):
"""Prune the audit logs of objects that no longer exist."""
help = "Prunes the audit logs of objects that no longer exist."
def add_arguments(self, parser):
self.add_argument_progress_bar_mixin(parser)
def handle(self, **options):
self.handle_progress_bar_mixin(**options)
def handle(self, *args, **options):
with transaction.atomic():
for log_entry in tqdm(LogEntry.objects.all(), disable=self.no_progress_bar):
for log_entry in self.track(
LogEntry.objects.all(),
description="Pruning audit logs...",
):
model_class = log_entry.content_type.model_class()
# use global_objects for SoftDeleteModel
objects = (
model_class.global_objects
if hasattr(model_class, "global_objects")
@@ -32,8 +26,8 @@ class Command(BaseCommand, ProgressBarMixin):
and not objects.filter(pk=log_entry.object_id).exists()
):
log_entry.delete()
tqdm.write(
self.style.NOTICE(
f"Deleted audit log entry for {model_class.__name__} #{log_entry.object_id}",
),
self.console.print(
f"Deleted audit log entry for "
f"{model_class.__name__} #{log_entry.object_id}",
style="yellow",
)

View File

@@ -1,4 +1,4 @@
# Generated by Django 5.2.7 on 2026-01-15 22:08
# Generated by Django 5.2.11 on 2026-03-03 16:27
import datetime
@@ -21,6 +21,207 @@ class Migration(migrations.Migration):
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
replaces = [
("documents", "0001_initial"),
("documents", "0002_auto_20151226_1316"),
("documents", "0003_sender"),
("documents", "0004_auto_20160114_1844"),
(
"documents",
"0004_auto_20160114_1844_squashed_0011_auto_20160303_1929",
),
("documents", "0005_auto_20160123_0313"),
("documents", "0006_auto_20160123_0430"),
("documents", "0007_auto_20160126_2114"),
("documents", "0008_document_file_type"),
("documents", "0009_auto_20160214_0040"),
("documents", "0010_log"),
("documents", "0011_auto_20160303_1929"),
("documents", "0012_auto_20160305_0040"),
("documents", "0013_auto_20160325_2111"),
("documents", "0014_document_checksum"),
("documents", "0015_add_insensitive_to_match"),
(
"documents",
"0015_add_insensitive_to_match_squashed_0018_auto_20170715_1712",
),
("documents", "0016_auto_20170325_1558"),
("documents", "0017_auto_20170512_0507"),
("documents", "0018_auto_20170715_1712"),
("documents", "0019_add_consumer_user"),
("documents", "0020_document_added"),
("documents", "0021_document_storage_type"),
("documents", "0022_auto_20181007_1420"),
("documents", "0023_document_current_filename"),
("documents", "1000_update_paperless_all"),
("documents", "1001_auto_20201109_1636"),
("documents", "1002_auto_20201111_1105"),
("documents", "1003_mime_types"),
("documents", "1004_sanity_check_schedule"),
("documents", "1005_checksums"),
("documents", "1006_auto_20201208_2209"),
(
"documents",
"1006_auto_20201208_2209_squashed_1011_auto_20210101_2340",
),
("documents", "1007_savedview_savedviewfilterrule"),
("documents", "1008_auto_20201216_1736"),
("documents", "1009_auto_20201216_2005"),
("documents", "1010_auto_20210101_2159"),
("documents", "1011_auto_20210101_2340"),
("documents", "1012_fix_archive_files"),
("documents", "1013_migrate_tag_colour"),
("documents", "1014_auto_20210228_1614"),
("documents", "1015_remove_null_characters"),
("documents", "1016_auto_20210317_1351"),
(
"documents",
"1016_auto_20210317_1351_squashed_1020_merge_20220518_1839",
),
("documents", "1017_alter_savedviewfilterrule_rule_type"),
("documents", "1018_alter_savedviewfilterrule_value"),
("documents", "1019_storagepath_document_storage_path"),
("documents", "1019_uisettings"),
("documents", "1020_merge_20220518_1839"),
("documents", "1021_webp_thumbnail_conversion"),
("documents", "1022_paperlesstask"),
(
"documents",
"1022_paperlesstask_squashed_1036_alter_savedviewfilterrule_rule_type",
),
("documents", "1023_add_comments"),
("documents", "1024_document_original_filename"),
("documents", "1025_alter_savedviewfilterrule_rule_type"),
("documents", "1026_transition_to_celery"),
(
"documents",
"1027_remove_paperlesstask_attempted_task_and_more",
),
(
"documents",
"1028_remove_paperlesstask_task_args_and_more",
),
("documents", "1029_alter_document_archive_serial_number"),
("documents", "1030_alter_paperlesstask_task_file_name"),
(
"documents",
"1031_remove_savedview_user_correspondent_owner_and_more",
),
(
"documents",
"1032_alter_correspondent_matching_algorithm_and_more",
),
(
"documents",
"1033_alter_documenttype_options_alter_tag_options_and_more",
),
("documents", "1034_alter_savedviewfilterrule_rule_type"),
("documents", "1035_rename_comment_note"),
("documents", "1036_alter_savedviewfilterrule_rule_type"),
("documents", "1037_webp_encrypted_thumbnail_conversion"),
("documents", "1038_sharelink"),
("documents", "1039_consumptiontemplate"),
(
"documents",
"1040_customfield_customfieldinstance_and_more",
),
("documents", "1041_alter_consumptiontemplate_sources"),
(
"documents",
"1042_consumptiontemplate_assign_custom_fields_and_more",
),
("documents", "1043_alter_savedviewfilterrule_rule_type"),
(
"documents",
"1044_workflow_workflowaction_workflowtrigger_and_more",
),
(
"documents",
"1045_alter_customfieldinstance_value_monetary",
),
(
"documents",
"1045_alter_customfieldinstance_value_monetary_squashed_1049_document_deleted_at_document_restored_at",
),
(
"documents",
"1046_workflowaction_remove_all_correspondents_and_more",
),
("documents", "1047_savedview_display_mode_and_more"),
("documents", "1048_alter_savedviewfilterrule_rule_type"),
(
"documents",
"1049_document_deleted_at_document_restored_at",
),
("documents", "1050_customfield_extra_data_and_more"),
(
"documents",
"1051_alter_correspondent_owner_alter_document_owner_and_more",
),
("documents", "1052_document_transaction_id"),
("documents", "1053_document_page_count"),
(
"documents",
"1054_customfieldinstance_value_monetary_amount_and_more",
),
("documents", "1055_alter_storagepath_path"),
(
"documents",
"1056_customfieldinstance_deleted_at_and_more",
),
("documents", "1057_paperlesstask_owner"),
(
"documents",
"1058_workflowtrigger_schedule_date_custom_field_and_more",
),
(
"documents",
"1059_workflowactionemail_workflowactionwebhook_and_more",
),
(
"documents",
"1060_alter_customfieldinstance_value_select",
),
("documents", "1061_workflowactionwebhook_as_json"),
("documents", "1062_alter_savedviewfilterrule_rule_type"),
(
"documents",
"1063_paperlesstask_type_alter_paperlesstask_task_name_and_more",
),
("documents", "1064_delete_log"),
(
"documents",
"1065_workflowaction_assign_custom_fields_values",
),
(
"documents",
"1066_alter_workflowtrigger_schedule_offset_days",
),
("documents", "1067_alter_document_created"),
("documents", "1068_alter_document_created"),
(
"documents",
"1069_workflowtrigger_filter_has_storage_path_and_more",
),
(
"documents",
"1070_customfieldinstance_value_long_text_and_more",
),
(
"documents",
"1071_tag_tn_ancestors_count_tag_tn_ancestors_pks_and_more",
),
(
"documents",
"1072_workflowtrigger_filter_custom_field_query_and_more",
),
("documents", "1073_migrate_workflow_title_jinja"),
(
"documents",
"1074_workflowrun_deleted_at_workflowrun_restored_at_and_more",
),
]
operations = [
migrations.CreateModel(
name="WorkflowActionEmail",
@@ -185,70 +386,6 @@ class Migration(migrations.Migration):
"abstract": False,
},
),
migrations.CreateModel(
name="CustomField",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
models.DateTimeField(
db_index=True,
default=django.utils.timezone.now,
editable=False,
verbose_name="created",
),
),
("name", models.CharField(max_length=128)),
(
"data_type",
models.CharField(
choices=[
("string", "String"),
("url", "URL"),
("date", "Date"),
("boolean", "Boolean"),
("integer", "Integer"),
("float", "Float"),
("monetary", "Monetary"),
("documentlink", "Document Link"),
("select", "Select"),
("longtext", "Long Text"),
],
editable=False,
max_length=50,
verbose_name="data type",
),
),
(
"extra_data",
models.JSONField(
blank=True,
help_text="Extra data for the custom field, such as select options",
null=True,
verbose_name="extra data",
),
),
],
options={
"verbose_name": "custom field",
"verbose_name_plural": "custom fields",
"ordering": ("created",),
"constraints": [
models.UniqueConstraint(
fields=("name",),
name="documents_customfield_unique_name",
),
],
},
),
migrations.CreateModel(
name="DocumentType",
fields=[
@@ -733,17 +870,6 @@ class Migration(migrations.Migration):
verbose_name="correspondent",
),
),
(
"owner",
models.ForeignKey(
blank=True,
default=None,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to=settings.AUTH_USER_MODEL,
verbose_name="owner",
),
),
(
"document_type",
models.ForeignKey(
@@ -767,12 +893,14 @@ class Migration(migrations.Migration):
),
),
(
"tags",
models.ManyToManyField(
"owner",
models.ForeignKey(
blank=True,
related_name="documents",
to="documents.tag",
verbose_name="tags",
default=None,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
to=settings.AUTH_USER_MODEL,
verbose_name="owner",
),
),
],
@@ -782,6 +910,140 @@ class Migration(migrations.Migration):
"ordering": ("-created",),
},
),
migrations.AddField(
model_name="document",
name="tags",
field=models.ManyToManyField(
blank=True,
related_name="documents",
to="documents.tag",
verbose_name="tags",
),
),
migrations.CreateModel(
name="Note",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("deleted_at", models.DateTimeField(blank=True, null=True)),
("restored_at", models.DateTimeField(blank=True, null=True)),
("transaction_id", models.UUIDField(blank=True, null=True)),
(
"note",
models.TextField(
blank=True,
help_text="Note for the document",
verbose_name="content",
),
),
(
"created",
models.DateTimeField(
db_index=True,
default=django.utils.timezone.now,
verbose_name="created",
),
),
(
"document",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
related_name="notes",
to="documents.document",
verbose_name="document",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="notes",
to=settings.AUTH_USER_MODEL,
verbose_name="user",
),
),
],
options={
"verbose_name": "note",
"verbose_name_plural": "notes",
"ordering": ("created",),
},
),
migrations.CreateModel(
name="CustomField",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
(
"created",
models.DateTimeField(
db_index=True,
default=django.utils.timezone.now,
editable=False,
verbose_name="created",
),
),
("name", models.CharField(max_length=128)),
(
"data_type",
models.CharField(
choices=[
("string", "String"),
("url", "URL"),
("date", "Date"),
("boolean", "Boolean"),
("integer", "Integer"),
("float", "Float"),
("monetary", "Monetary"),
("documentlink", "Document Link"),
("select", "Select"),
("longtext", "Long Text"),
],
editable=False,
max_length=50,
verbose_name="data type",
),
),
(
"extra_data",
models.JSONField(
blank=True,
help_text="Extra data for the custom field, such as select options",
null=True,
verbose_name="extra data",
),
),
],
options={
"verbose_name": "custom field",
"verbose_name_plural": "custom fields",
"ordering": ("created",),
"constraints": [
models.UniqueConstraint(
fields=("name",),
name="documents_customfield_unique_name",
),
],
},
),
migrations.CreateModel(
name="CustomFieldInstance",
fields=[
@@ -880,66 +1142,6 @@ class Migration(migrations.Migration):
"ordering": ("created",),
},
),
migrations.CreateModel(
name="Note",
fields=[
(
"id",
models.AutoField(
auto_created=True,
primary_key=True,
serialize=False,
verbose_name="ID",
),
),
("deleted_at", models.DateTimeField(blank=True, null=True)),
("restored_at", models.DateTimeField(blank=True, null=True)),
("transaction_id", models.UUIDField(blank=True, null=True)),
(
"note",
models.TextField(
blank=True,
help_text="Note for the document",
verbose_name="content",
),
),
(
"created",
models.DateTimeField(
db_index=True,
default=django.utils.timezone.now,
verbose_name="created",
),
),
(
"document",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.CASCADE,
related_name="notes",
to="documents.document",
verbose_name="document",
),
),
(
"user",
models.ForeignKey(
blank=True,
null=True,
on_delete=django.db.models.deletion.SET_NULL,
related_name="notes",
to=settings.AUTH_USER_MODEL,
verbose_name="user",
),
),
],
options={
"verbose_name": "note",
"verbose_name_plural": "notes",
"ordering": ("created",),
},
),
migrations.CreateModel(
name="PaperlessTask",
fields=[
@@ -986,7 +1188,6 @@ class Migration(migrations.Migration):
("train_classifier", "Train Classifier"),
("check_sanity", "Check Sanity"),
("index_optimize", "Index Optimize"),
("llmindex_update", "LLM Index Update"),
],
help_text="Name of the task that was run",
max_length=255,
@@ -1380,6 +1581,7 @@ class Migration(migrations.Migration):
verbose_name="Workflow Action Type",
),
),
("order", models.PositiveIntegerField(default=0, verbose_name="order")),
(
"assign_title",
models.TextField(

View File

@@ -1,4 +1,4 @@
# Generated by Django 5.2.9 on 2026-01-20 18:46
# Generated by Django 5.2.11 on 2026-03-03 16:27
import django.db.models.deletion
from django.db import migrations
@@ -9,8 +9,14 @@ class Migration(migrations.Migration):
initial = True
dependencies = [
("documents", "0001_initial"),
("paperless_mail", "0001_initial"),
("documents", "0001_squashed"),
("paperless_mail", "0001_squashed"),
]
# This migration needs a "replaces", but it doesn't matter which.
# Chose the last 2.20.x migration
replaces = [
("documents", "1075_workflowaction_order"),
]
operations = [

View File

@@ -6,7 +6,7 @@ from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0002_initial"),
("documents", "0002_squashed"),
]
operations = [

View File

@@ -0,0 +1,30 @@
# Generated by Django 5.2.11 on 2026-03-03 16:42
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0013_document_root_document"),
]
operations = [
migrations.AlterField(
model_name="paperlesstask",
name="task_name",
field=models.CharField(
choices=[
("consume_file", "Consume File"),
("train_classifier", "Train Classifier"),
("check_sanity", "Check Sanity"),
("index_optimize", "Index Optimize"),
("llmindex_update", "LLM Index Update"),
],
help_text="Name of the task that was run",
max_length=255,
null=True,
verbose_name="Task Name",
),
),
]

View File

@@ -0,0 +1,153 @@
# Generated by Django 5.2.11 on 2026-02-20 22:05
from collections import defaultdict
from django.db import migrations
from django.db import models
# from src-ui/src/app/data/ui-settings.ts
SAVED_VIEWS_KEY = "saved_views"
DASHBOARD_VIEWS_VISIBLE_IDS_KEY = "dashboard_views_visible_ids"
SIDEBAR_VIEWS_VISIBLE_IDS_KEY = "sidebar_views_visible_ids"
def _parse_visible_ids(raw_value) -> set[int]:
"""Return integer SavedView IDs parsed from a JSON list value."""
if not isinstance(raw_value, list):
return set()
parsed_ids = set()
for raw_id in raw_value:
raw_id_string = str(raw_id)
if raw_id_string.isdigit():
parsed_ids.add(int(raw_id_string))
return parsed_ids
def _set_default_visibility_ids(apps, schema_editor):
"""
Move SavedView visibility from boolean model props into JSON UiSettings.settings.saved_views, specifically:
settings.saved_views.dashboard_views_visible_ids
settings.saved_views.sidebar_views_visible_ids
"""
SavedView = apps.get_model("documents", "SavedView")
UiSettings = apps.get_model("documents", "UiSettings")
User = apps.get_model("auth", "User")
dashboard_visible_ids_by_owner: defaultdict[int, list[int]] = defaultdict(list)
for owner_id, view_id in SavedView.objects.filter(
owner__isnull=False,
show_on_dashboard=True,
).values_list("owner_id", "id"):
dashboard_visible_ids_by_owner[owner_id].append(view_id)
sidebar_visible_ids_by_owner: defaultdict[int, list[int]] = defaultdict(list)
for owner_id, view_id in SavedView.objects.filter(
owner__isnull=False,
show_in_sidebar=True,
).values_list("owner_id", "id"):
sidebar_visible_ids_by_owner[owner_id].append(view_id)
for user in User.objects.all():
ui_settings, _ = UiSettings.objects.get_or_create(
user=user,
defaults={"settings": {}},
)
current_settings = ui_settings.settings
if not isinstance(current_settings, dict):
current_settings = {}
changed = False
saved_views_settings = current_settings.get(SAVED_VIEWS_KEY)
if not isinstance(saved_views_settings, dict):
saved_views_settings = {}
changed = True
if saved_views_settings.get(DASHBOARD_VIEWS_VISIBLE_IDS_KEY) is None:
saved_views_settings[DASHBOARD_VIEWS_VISIBLE_IDS_KEY] = (
dashboard_visible_ids_by_owner.get(user.id, [])
)
changed = True
if saved_views_settings.get(SIDEBAR_VIEWS_VISIBLE_IDS_KEY) is None:
saved_views_settings[SIDEBAR_VIEWS_VISIBLE_IDS_KEY] = (
sidebar_visible_ids_by_owner.get(user.id, [])
)
changed = True
current_settings[SAVED_VIEWS_KEY] = saved_views_settings
if changed:
ui_settings.settings = current_settings
ui_settings.save(update_fields=["settings"])
def _restore_visibility_fields(apps, schema_editor):
SavedView = apps.get_model("documents", "SavedView")
UiSettings = apps.get_model("documents", "UiSettings")
dashboard_visible_ids_by_owner: dict[int, set[int]] = {}
sidebar_visible_ids_by_owner: dict[int, set[int]] = {}
for ui_settings in UiSettings.objects.all():
current_settings = ui_settings.settings
if not isinstance(current_settings, dict):
continue
saved_views_settings = current_settings.get(SAVED_VIEWS_KEY)
if not isinstance(saved_views_settings, dict):
saved_views_settings = {}
dashboard_visible_ids_by_owner[ui_settings.user_id] = _parse_visible_ids(
saved_views_settings.get(DASHBOARD_VIEWS_VISIBLE_IDS_KEY),
)
sidebar_visible_ids_by_owner[ui_settings.user_id] = _parse_visible_ids(
saved_views_settings.get(SIDEBAR_VIEWS_VISIBLE_IDS_KEY),
)
SavedView.objects.update(show_on_dashboard=False, show_in_sidebar=False)
for owner_id, dashboard_visible_ids in dashboard_visible_ids_by_owner.items():
if not dashboard_visible_ids:
continue
SavedView.objects.filter(
owner_id=owner_id,
id__in=dashboard_visible_ids,
).update(
show_on_dashboard=True,
)
for owner_id, sidebar_visible_ids in sidebar_visible_ids_by_owner.items():
if not sidebar_visible_ids:
continue
SavedView.objects.filter(owner_id=owner_id, id__in=sidebar_visible_ids).update(
show_in_sidebar=True,
)
class Migration(migrations.Migration):
dependencies = [
("documents", "0014_alter_paperlesstask_task_name"),
]
operations = [
migrations.AlterField(
model_name="savedview",
name="show_on_dashboard",
field=models.BooleanField(default=False, verbose_name="show on dashboard"),
),
migrations.AlterField(
model_name="savedview",
name="show_in_sidebar",
field=models.BooleanField(default=False, verbose_name="show in sidebar"),
),
migrations.RunPython(
_set_default_visibility_ids,
reverse_code=_restore_visibility_fields,
),
migrations.RemoveField(
model_name="savedview",
name="show_on_dashboard",
),
migrations.RemoveField(
model_name="savedview",
name="show_in_sidebar",
),
]

View File

@@ -0,0 +1,37 @@
# Generated by Django 5.2.11 on 2026-03-02 17:48
from django.conf import settings
from django.db import migrations
from django.db import models
class Migration(migrations.Migration):
dependencies = [
("documents", "0015_savedview_visibility_to_ui_settings"),
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
]
operations = [
migrations.AddField(
model_name="document",
name="version_index",
field=models.PositiveIntegerField(
blank=True,
db_index=True,
help_text="Index of this version within the root document.",
null=True,
verbose_name="version index",
),
),
migrations.AddConstraint(
model_name="document",
constraint=models.UniqueConstraint(
condition=models.Q(
("root_document__isnull", False),
("version_index__isnull", False),
),
fields=("root_document", "version_index"),
name="documents_document_root_version_index_uniq",
),
),
]

View File

@@ -75,7 +75,7 @@ class MatchingModel(ModelWithOwner):
is_insensitive = models.BooleanField(_("is insensitive"), default=True)
class Meta:
class Meta(ModelWithOwner.Meta):
abstract = True
ordering = ("name",)
constraints = [
@@ -156,6 +156,8 @@ class StoragePath(MatchingModel):
class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-missing]
MAX_STORED_FILENAME_LENGTH: Final[int] = 1024
correspondent = models.ForeignKey(
Correspondent,
blank=True,
@@ -262,7 +264,7 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
filename = models.FilePathField(
_("filename"),
max_length=1024,
max_length=MAX_STORED_FILENAME_LENGTH,
editable=False,
default=None,
unique=True,
@@ -272,7 +274,7 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
archive_filename = models.FilePathField(
_("archive filename"),
max_length=1024,
max_length=MAX_STORED_FILENAME_LENGTH,
editable=False,
default=None,
unique=True,
@@ -282,7 +284,7 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
original_filename = models.CharField(
_("original filename"),
max_length=1024,
max_length=MAX_STORED_FILENAME_LENGTH,
editable=False,
default=None,
unique=False,
@@ -317,6 +319,14 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
verbose_name=_("root document for this version"),
)
version_index = models.PositiveIntegerField(
_("version index"),
blank=True,
null=True,
db_index=True,
help_text=_("Index of this version within the root document."),
)
version_label = models.CharField(
_("version label"),
max_length=64,
@@ -329,6 +339,16 @@ class Document(SoftDeleteModel, ModelWithOwner): # type: ignore[django-manager-
ordering = ("-created",)
verbose_name = _("document")
verbose_name_plural = _("documents")
constraints = [
models.UniqueConstraint(
fields=["root_document", "version_index"],
condition=models.Q(
root_document__isnull=False,
version_index__isnull=False,
),
name="documents_document_root_version_index_uniq",
),
]
def __str__(self) -> str:
created = self.created.isoformat()
@@ -473,13 +493,6 @@ class SavedView(ModelWithOwner):
name = models.CharField(_("name"), max_length=128)
show_on_dashboard = models.BooleanField(
_("show on dashboard"),
)
show_in_sidebar = models.BooleanField(
_("show in sidebar"),
)
sort_field = models.CharField(
_("sort field"),
max_length=128,

View File

@@ -5,11 +5,7 @@ from abc import abstractmethod
from collections.abc import Iterator
from dataclasses import dataclass
from types import TracebackType
try:
from typing import Self
except ImportError:
from typing_extensions import Self
from typing import Self
import dateparser

View File

@@ -1,4 +1,5 @@
import enum
from collections.abc import Mapping
from typing import TYPE_CHECKING
from asgiref.sync import async_to_sync
@@ -8,7 +9,7 @@ if TYPE_CHECKING:
from channels_redis.pubsub import RedisPubSubChannelLayer
class ProgressStatusOptions(str, enum.Enum):
class ProgressStatusOptions(enum.StrEnum):
STARTED = "STARTED"
WORKING = "WORKING"
SUCCESS = "SUCCESS"
@@ -47,7 +48,7 @@ class BaseStatusManager:
async_to_sync(self._channel.flush)
self._channel = None
def send(self, payload: dict[str, str | int | None]) -> None:
def send(self, payload: Mapping[str, object]) -> None:
# Ensure the layer is open
self.open()
@@ -73,26 +74,28 @@ class ProgressManager(BaseStatusManager):
max_progress: int,
extra_args: dict[str, str | int | None] | None = None,
) -> None:
payload = {
"type": "status_update",
"data": {
"filename": self.filename,
"task_id": self.task_id,
"current_progress": current_progress,
"max_progress": max_progress,
"status": status,
"message": message,
},
data: dict[str, object] = {
"filename": self.filename,
"task_id": self.task_id,
"current_progress": current_progress,
"max_progress": max_progress,
"status": status,
"message": message,
}
if extra_args is not None:
payload["data"].update(extra_args)
data.update(extra_args)
payload: dict[str, object] = {
"type": "status_update",
"data": data,
}
self.send(payload)
class DocumentsStatusManager(BaseStatusManager):
def send_documents_deleted(self, documents: list[int]) -> None:
payload = {
payload: dict[str, object] = {
"type": "documents_deleted",
"data": {
"documents": documents,
@@ -100,3 +103,25 @@ class DocumentsStatusManager(BaseStatusManager):
}
self.send(payload)
def send_document_updated(
self,
*,
document_id: int,
modified: str,
owner_id: int | None = None,
users_can_view: list[int] | None = None,
groups_can_view: list[int] | None = None,
) -> None:
payload: dict[str, object] = {
"type": "document_updated",
"data": {
"document_id": document_id,
"modified": modified,
"owner_id": owner_id,
"users_can_view": users_can_view or [],
"groups_can_view": groups_can_view or [],
},
}
self.send(payload)

View File

@@ -1,80 +1,174 @@
"""
Sanity checker for the Paperless-ngx document archive.
Verifies that all documents have valid files, correct checksums,
and consistent metadata. Reports orphaned files in the media directory.
Progress display is the caller's responsibility -- pass an ``iter_wrapper``
to wrap the document queryset (e.g., with a progress bar). The default
is an identity function that adds no overhead.
"""
from __future__ import annotations
import hashlib
import logging
import uuid
from collections import defaultdict
from collections.abc import Callable
from collections.abc import Iterable
from collections.abc import Iterator
from pathlib import Path
from typing import TYPE_CHECKING
from typing import Final
from typing import TypedDict
from typing import TypeVar
from celery import states
from django.conf import settings
from django.utils import timezone
from tqdm import tqdm
from documents.models import Document
from documents.models import PaperlessTask
from paperless.config import GeneralConfig
logger = logging.getLogger("paperless.sanity_checker")
_T = TypeVar("_T")
IterWrapper = Callable[[Iterable[_T]], Iterable[_T]]
class MessageEntry(TypedDict):
"""A single sanity check message with its severity level."""
level: int
message: str
def _identity(iterable: Iterable[_T]) -> Iterable[_T]:
"""Pass through an iterable unchanged (default iter_wrapper)."""
return iterable
class SanityCheckMessages:
def __init__(self) -> None:
self._messages: dict[int, list[dict]] = defaultdict(list)
self.has_error = False
self.has_warning = False
"""Collects sanity check messages grouped by document primary key.
def error(self, doc_pk, message) -> None:
Messages are categorized as error, warning, or info. ``None`` is used
as the key for messages not associated with a specific document
(e.g., orphaned files).
"""
def __init__(self) -> None:
self._messages: dict[int | None, list[MessageEntry]] = defaultdict(list)
self.has_error: bool = False
self.has_warning: bool = False
self.has_info: bool = False
self.document_count: int = 0
self.document_error_count: int = 0
self.document_warning_count: int = 0
self.document_info_count: int = 0
self.global_warning_count: int = 0
# -- Recording ----------------------------------------------------------
def error(self, doc_pk: int | None, message: str) -> None:
self._messages[doc_pk].append({"level": logging.ERROR, "message": message})
self.has_error = True
if doc_pk is not None:
self.document_count += 1
self.document_error_count += 1
def warning(self, doc_pk, message) -> None:
def warning(self, doc_pk: int | None, message: str) -> None:
self._messages[doc_pk].append({"level": logging.WARNING, "message": message})
self.has_warning = True
def info(self, doc_pk, message) -> None:
if doc_pk is not None:
self.document_count += 1
self.document_warning_count += 1
else:
# This is the only type of global message we do right now
self.global_warning_count += 1
def info(self, doc_pk: int | None, message: str) -> None:
self._messages[doc_pk].append({"level": logging.INFO, "message": message})
self.has_info = True
if doc_pk is not None:
self.document_count += 1
self.document_info_count += 1
# -- Iteration / query --------------------------------------------------
def document_pks(self) -> list[int | None]:
"""Return all document PKs (including None for global messages)."""
return list(self._messages.keys())
def iter_messages(self) -> Iterator[tuple[int | None, list[MessageEntry]]]:
"""Iterate over (doc_pk, messages) pairs."""
yield from self._messages.items()
def __getitem__(self, item: int | None) -> list[MessageEntry]:
return self._messages[item]
# -- Summarize Helpers --------------------------------------------------
@property
def has_global_issues(self) -> bool:
return None in self._messages
@property
def total_issue_count(self) -> int:
"""Total number of error and warning messages across all documents and global."""
return (
self.document_error_count
+ self.document_warning_count
+ self.global_warning_count
)
# -- Logging output (used by Celery task path) --------------------------
def log_messages(self) -> None:
logger = logging.getLogger("paperless.sanity_checker")
"""Write all messages to the ``paperless.sanity_checker`` logger.
This is the output path for headless / Celery execution.
Management commands use Rich rendering instead.
"""
if len(self._messages) == 0:
logger.info("Sanity checker detected no issues.")
else:
# Query once
all_docs = Document.global_objects.all()
return
for doc_pk in self._messages:
if doc_pk is not None:
doc = all_docs.get(pk=doc_pk)
logger.info(
f"Detected following issue(s) with document #{doc.pk},"
f" titled {doc.title}",
)
for msg in self._messages[doc_pk]:
logger.log(msg["level"], msg["message"])
doc_pks = [pk for pk in self._messages if pk is not None]
titles: dict[int, str] = {}
if doc_pks:
titles = dict(
Document.global_objects.filter(pk__in=doc_pks)
.only("pk", "title")
.values_list("pk", "title"),
)
def __len__(self):
return len(self._messages)
def __getitem__(self, item):
return self._messages[item]
for doc_pk, entries in self._messages.items():
if doc_pk is not None:
title = titles.get(doc_pk, "Unknown")
logger.info(
"Detected following issue(s) with document #%s, titled %s",
doc_pk,
title,
)
for msg in entries:
logger.log(msg["level"], msg["message"])
class SanityCheckFailedException(Exception):
pass
def check_sanity(*, progress=False, scheduled=True) -> SanityCheckMessages:
paperless_task = PaperlessTask.objects.create(
task_id=uuid.uuid4(),
type=PaperlessTask.TaskType.SCHEDULED_TASK
if scheduled
else PaperlessTask.TaskType.MANUAL_TASK,
task_name=PaperlessTask.TaskName.CHECK_SANITY,
status=states.STARTED,
date_created=timezone.now(),
date_started=timezone.now(),
)
messages = SanityCheckMessages()
# ---------------------------------------------------------------------------
# Internal helpers
# ---------------------------------------------------------------------------
def _build_present_files() -> set[Path]:
"""Collect all files in MEDIA_ROOT, excluding directories and ignorable files."""
present_files = {
x.resolve()
for x in Path(settings.MEDIA_ROOT).glob("**/*")
@@ -82,95 +176,178 @@ def check_sanity(*, progress=False, scheduled=True) -> SanityCheckMessages:
}
lockfile = Path(settings.MEDIA_LOCK).resolve()
if lockfile in present_files:
present_files.remove(lockfile)
present_files.discard(lockfile)
general_config = GeneralConfig()
app_logo = general_config.app_logo or settings.APP_LOGO
if app_logo:
logo_file = Path(settings.MEDIA_ROOT / Path(app_logo.lstrip("/"))).resolve()
if logo_file in present_files:
present_files.remove(logo_file)
present_files.discard(logo_file)
for doc in tqdm(Document.global_objects.all(), disable=not progress):
# Check sanity of the thumbnail
thumbnail_path: Final[Path] = Path(doc.thumbnail_path).resolve()
if not thumbnail_path.exists() or not thumbnail_path.is_file():
messages.error(doc.pk, "Thumbnail of document does not exist.")
else:
if thumbnail_path in present_files:
present_files.remove(thumbnail_path)
try:
_ = thumbnail_path.read_bytes()
except OSError as e:
messages.error(doc.pk, f"Cannot read thumbnail file of document: {e}")
return present_files
# Check sanity of the original file
# TODO: extract method
source_path: Final[Path] = Path(doc.source_path).resolve()
if not source_path.exists() or not source_path.is_file():
messages.error(doc.pk, "Original of document does not exist.")
else:
if source_path in present_files:
present_files.remove(source_path)
try:
checksum = hashlib.md5(source_path.read_bytes()).hexdigest()
except OSError as e:
messages.error(doc.pk, f"Cannot read original file of document: {e}")
else:
if checksum != doc.checksum:
messages.error(
doc.pk,
"Checksum mismatch. "
f"Stored: {doc.checksum}, actual: {checksum}.",
)
# Check sanity of the archive file.
if doc.archive_checksum is not None and doc.archive_filename is None:
def _check_thumbnail(
doc: Document,
messages: SanityCheckMessages,
present_files: set[Path],
) -> None:
"""Verify the thumbnail exists and is readable."""
thumbnail_path: Final[Path] = Path(doc.thumbnail_path).resolve()
if not thumbnail_path.exists() or not thumbnail_path.is_file():
messages.error(doc.pk, "Thumbnail of document does not exist.")
return
present_files.discard(thumbnail_path)
try:
_ = thumbnail_path.read_bytes()
except OSError as e:
messages.error(doc.pk, f"Cannot read thumbnail file of document: {e}")
def _check_original(
doc: Document,
messages: SanityCheckMessages,
present_files: set[Path],
) -> None:
"""Verify the original file exists, is readable, and has matching checksum."""
source_path: Final[Path] = Path(doc.source_path).resolve()
if not source_path.exists() or not source_path.is_file():
messages.error(doc.pk, "Original of document does not exist.")
return
present_files.discard(source_path)
try:
checksum = hashlib.md5(source_path.read_bytes()).hexdigest()
except OSError as e:
messages.error(doc.pk, f"Cannot read original file of document: {e}")
else:
if checksum != doc.checksum:
messages.error(
doc.pk,
"Document has an archive file checksum, but no archive filename.",
f"Checksum mismatch. Stored: {doc.checksum}, actual: {checksum}.",
)
elif doc.archive_checksum is None and doc.archive_filename is not None:
def _check_archive(
doc: Document,
messages: SanityCheckMessages,
present_files: set[Path],
) -> None:
"""Verify archive file consistency: checksum/filename pairing and file integrity."""
if doc.archive_checksum is not None and doc.archive_filename is None:
messages.error(
doc.pk,
"Document has an archive file checksum, but no archive filename.",
)
elif doc.archive_checksum is None and doc.archive_filename is not None:
messages.error(
doc.pk,
"Document has an archive file, but its checksum is missing.",
)
elif doc.has_archive_version:
if TYPE_CHECKING:
assert isinstance(doc.archive_path, Path)
archive_path: Final[Path] = Path(doc.archive_path).resolve()
if not archive_path.exists() or not archive_path.is_file():
messages.error(doc.pk, "Archived version of document does not exist.")
return
present_files.discard(archive_path)
try:
checksum = hashlib.md5(archive_path.read_bytes()).hexdigest()
except OSError as e:
messages.error(
doc.pk,
"Document has an archive file, but its checksum is missing.",
f"Cannot read archive file of document: {e}",
)
elif doc.has_archive_version:
archive_path: Final[Path] = Path(doc.archive_path).resolve()
if not archive_path.exists() or not archive_path.is_file():
messages.error(doc.pk, "Archived version of document does not exist.")
else:
if archive_path in present_files:
present_files.remove(archive_path)
try:
checksum = hashlib.md5(archive_path.read_bytes()).hexdigest()
except OSError as e:
messages.error(
doc.pk,
f"Cannot read archive file of document : {e}",
)
else:
if checksum != doc.archive_checksum:
messages.error(
doc.pk,
"Checksum mismatch of archived document. "
f"Stored: {doc.archive_checksum}, "
f"actual: {checksum}.",
)
else:
if checksum != doc.archive_checksum:
messages.error(
doc.pk,
"Checksum mismatch of archived document. "
f"Stored: {doc.archive_checksum}, actual: {checksum}.",
)
# other document checks
if not doc.content:
messages.info(doc.pk, "Document contains no OCR data")
def _check_content(doc: Document, messages: SanityCheckMessages) -> None:
"""Flag documents with no OCR content."""
if not doc.content:
messages.info(doc.pk, "Document contains no OCR data")
def _check_document(
doc: Document,
messages: SanityCheckMessages,
present_files: set[Path],
) -> None:
"""Run all checks for a single document."""
_check_thumbnail(doc, messages, present_files)
_check_original(doc, messages, present_files)
_check_archive(doc, messages, present_files)
_check_content(doc, messages)
# ---------------------------------------------------------------------------
# Public entry point
# ---------------------------------------------------------------------------
def check_sanity(
*,
scheduled: bool = True,
iter_wrapper: IterWrapper[Document] = _identity,
) -> SanityCheckMessages:
"""Run a full sanity check on the document archive.
Args:
scheduled: Whether this is a scheduled (automatic) or manual check.
Controls the task type recorded in the database.
iter_wrapper: A callable that wraps the document iterable, e.g.,
for progress bar display. Defaults to identity (no wrapping).
Returns:
A SanityCheckMessages instance containing all detected issues.
"""
paperless_task = PaperlessTask.objects.create(
task_id=uuid.uuid4(),
type=(
PaperlessTask.TaskType.SCHEDULED_TASK
if scheduled
else PaperlessTask.TaskType.MANUAL_TASK
),
task_name=PaperlessTask.TaskName.CHECK_SANITY,
status=states.STARTED,
date_created=timezone.now(),
date_started=timezone.now(),
)
messages = SanityCheckMessages()
present_files = _build_present_files()
documents = Document.global_objects.all()
for doc in iter_wrapper(documents):
_check_document(doc, messages, present_files)
for extra_file in present_files:
messages.warning(None, f"Orphaned file in media dir: {extra_file}")
paperless_task.status = states.SUCCESS if not messages.has_error else states.FAILURE
# result is concatenated messages
paperless_task.result = f"{len(messages)} issues found."
if messages.has_error:
paperless_task.result += " Check logs for details."
if messages.total_issue_count == 0:
paperless_task.result = "No issues found."
else:
parts: list[str] = []
if messages.document_error_count:
parts.append(f"{messages.document_error_count} document(s) with errors")
if messages.document_warning_count:
parts.append(f"{messages.document_warning_count} document(s) with warnings")
if messages.global_warning_count:
parts.append(f"{messages.global_warning_count} global warning(s)")
paperless_task.result = ", ".join(parts) + " found."
if messages.has_error:
paperless_task.result += " Check logs for details."
paperless_task.date_done = timezone.now()
paperless_task.save(update_fields=["status", "result", "date_done"])
return messages

View File

@@ -80,6 +80,7 @@ from documents.parsers import is_mime_type_supported
from documents.permissions import get_document_count_filter_for_user
from documents.permissions import get_groups_with_only_permission
from documents.permissions import get_objects_for_user_owner_aware
from documents.permissions import has_perms_owner_aware
from documents.permissions import set_permissions_for_object
from documents.regex import validate_regex_pattern
from documents.templating.filepath import validate_filepath_template_and_render
@@ -1427,8 +1428,6 @@ class SavedViewSerializer(OwnedObjectSerializer):
fields = [
"id",
"name",
"show_on_dashboard",
"show_in_sidebar",
"sort_field",
"sort_reverse",
"filter_rules",
@@ -1438,6 +1437,7 @@ class SavedViewSerializer(OwnedObjectSerializer):
"owner",
"permissions",
"user_can_change",
"set_permissions",
]
def validate(self, attrs):
@@ -1475,7 +1475,7 @@ class SavedViewSerializer(OwnedObjectSerializer):
and len(validated_data["display_fields"]) == 0
):
validated_data["display_fields"] = None
super().update(instance, validated_data)
instance = super().update(instance, validated_data)
if rules_data is not None:
SavedViewFilterRule.objects.filter(saved_view=instance).delete()
for rule_data in rules_data:
@@ -1487,7 +1487,7 @@ class SavedViewSerializer(OwnedObjectSerializer):
if "user" in validated_data:
# backwards compatibility
validated_data["owner"] = validated_data.pop("user")
saved_view = SavedView.objects.create(**validated_data)
saved_view = super().create(validated_data)
for rule_data in rules_data:
SavedViewFilterRule.objects.create(saved_view=saved_view, **rule_data)
return saved_view
@@ -2321,6 +2321,17 @@ class ShareLinkSerializer(OwnedObjectSerializer):
validated_data["slug"] = get_random_string(50)
return super().create(validated_data)
def validate_document(self, document):
if self.user is not None and has_perms_owner_aware(
self.user,
"view_document",
document,
):
return document
raise PermissionDenied(
_("Insufficient permissions."),
)
class ShareLinkBundleSerializer(OwnedObjectSerializer):
document_ids = serializers.ListField(

View File

@@ -4,6 +4,7 @@ import logging
import shutil
from pathlib import Path
from typing import TYPE_CHECKING
from typing import Any
from celery import shared_task
from celery import states
@@ -23,6 +24,7 @@ from django.db.models import Q
from django.dispatch import receiver
from django.utils import timezone
from filelock import FileLock
from rest_framework import serializers
from documents import matching
from documents.caching import clear_document_caches
@@ -32,12 +34,14 @@ from documents.file_handling import create_source_path_directory
from documents.file_handling import delete_empty_directories
from documents.file_handling import generate_filename
from documents.file_handling import generate_unique_filename
from documents.models import Correspondent
from documents.models import CustomField
from documents.models import CustomFieldInstance
from documents.models import Document
from documents.models import MatchingModel
from documents.models import DocumentType
from documents.models import PaperlessTask
from documents.models import SavedView
from documents.models import StoragePath
from documents.models import Tag
from documents.models import UiSettings
from documents.models import Workflow
@@ -45,6 +49,7 @@ from documents.models import WorkflowAction
from documents.models import WorkflowRun
from documents.models import WorkflowTrigger
from documents.permissions import get_objects_for_user_owner_aware
from documents.plugins.helpers import DocumentsStatusManager
from documents.templating.utils import convert_format_str_to_template_format
from documents.workflows.actions import build_workflow_action_context
from documents.workflows.actions import execute_email_action
@@ -66,6 +71,7 @@ if TYPE_CHECKING:
from documents.data_models import DocumentMetadataOverrides
logger = logging.getLogger("paperless.handlers")
DRF_DATETIME_FIELD = serializers.DateTimeField()
def add_inbox_tags(sender, document: Document, logging_group=None, **kwargs) -> None:
@@ -81,47 +87,41 @@ def add_inbox_tags(sender, document: Document, logging_group=None, **kwargs) ->
document.add_nested_tags(inbox_tags)
def _suggestion_printer(
stdout,
style_func,
suggestion_type: str,
document: Document,
selected: MatchingModel,
base_url: str | None = None,
) -> None:
"""
Smaller helper to reduce duplication when just outputting suggestions to the console
"""
doc_str = str(document)
if base_url is not None:
stdout.write(style_func.SUCCESS(doc_str))
stdout.write(style_func.SUCCESS(f"{base_url}/documents/{document.pk}"))
else:
stdout.write(style_func.SUCCESS(f"{doc_str} [{document.pk}]"))
stdout.write(f"Suggest {suggestion_type}: {selected}")
def set_correspondent(
sender,
sender: object,
document: Document,
*,
logging_group=None,
logging_group: object = None,
classifier: DocumentClassifier | None = None,
replace=False,
use_first=True,
suggest=False,
base_url=None,
stdout=None,
style_func=None,
**kwargs,
) -> None:
replace: bool = False,
use_first: bool = True,
dry_run: bool = False,
**kwargs: Any,
) -> Correspondent | None:
"""
Assign a correspondent to a document based on classifier results.
Args:
document: The document to classify.
logging_group: Optional logging group for structured log output.
classifier: The trained classifier. If None, only rule-based matching runs.
replace: If True, overwrite an existing correspondent assignment.
use_first: If True, pick the first match when multiple correspondents
match. If False, skip assignment when multiple match.
dry_run: If True, compute and return the selection without saving.
**kwargs: Absorbed for Django signal compatibility (e.g. sender, signal).
Returns:
The correspondent that was (or would be) assigned, or None if no match
was found or assignment was skipped.
"""
if document.correspondent and not replace:
return
return None
potential_correspondents = matching.match_correspondents(document, classifier)
potential_count = len(potential_correspondents)
selected = potential_correspondents[0] if potential_correspondents else None
if potential_count > 1:
if use_first:
logger.debug(
@@ -135,49 +135,53 @@ def set_correspondent(
f"not assigning any correspondent",
extra={"group": logging_group},
)
return
return None
if selected or replace:
if suggest:
_suggestion_printer(
stdout,
style_func,
"correspondent",
document,
selected,
base_url,
)
else:
logger.info(
f"Assigning correspondent {selected} to {document}",
extra={"group": logging_group},
)
if (selected or replace) and not dry_run:
logger.info(
f"Assigning correspondent {selected} to {document}",
extra={"group": logging_group},
)
document.correspondent = selected
document.save(update_fields=("correspondent",))
document.correspondent = selected
document.save(update_fields=("correspondent",))
return selected
def set_document_type(
sender,
sender: object,
document: Document,
*,
logging_group=None,
logging_group: object = None,
classifier: DocumentClassifier | None = None,
replace=False,
use_first=True,
suggest=False,
base_url=None,
stdout=None,
style_func=None,
**kwargs,
) -> None:
replace: bool = False,
use_first: bool = True,
dry_run: bool = False,
**kwargs: Any,
) -> DocumentType | None:
"""
Assign a document type to a document based on classifier results.
Args:
document: The document to classify.
logging_group: Optional logging group for structured log output.
classifier: The trained classifier. If None, only rule-based matching runs.
replace: If True, overwrite an existing document type assignment.
use_first: If True, pick the first match when multiple types match.
If False, skip assignment when multiple match.
dry_run: If True, compute and return the selection without saving.
**kwargs: Absorbed for Django signal compatibility (e.g. sender, signal).
Returns:
The document type that was (or would be) assigned, or None if no match
was found or assignment was skipped.
"""
if document.document_type and not replace:
return
return None
potential_document_type = matching.match_document_types(document, classifier)
potential_count = len(potential_document_type)
selected = potential_document_type[0] if potential_document_type else None
potential_document_types = matching.match_document_types(document, classifier)
potential_count = len(potential_document_types)
selected = potential_document_types[0] if potential_document_types else None
if potential_count > 1:
if use_first:
@@ -192,42 +196,64 @@ def set_document_type(
f"not assigning any document type",
extra={"group": logging_group},
)
return
return None
if selected or replace:
if suggest:
_suggestion_printer(
stdout,
style_func,
"document type",
document,
selected,
base_url,
)
else:
logger.info(
f"Assigning document type {selected} to {document}",
extra={"group": logging_group},
)
if (selected or replace) and not dry_run:
logger.info(
f"Assigning document type {selected} to {document}",
extra={"group": logging_group},
)
document.document_type = selected
document.save(update_fields=("document_type",))
document.document_type = selected
document.save(update_fields=("document_type",))
return selected
def set_tags(
sender,
sender: object,
document: Document,
*,
logging_group=None,
logging_group: object = None,
classifier: DocumentClassifier | None = None,
replace=False,
suggest=False,
base_url=None,
stdout=None,
style_func=None,
**kwargs,
) -> None:
replace: bool = False,
dry_run: bool = False,
**kwargs: Any,
) -> tuple[set[Tag], set[Tag]]:
"""
Assign tags to a document based on classifier results.
When replace=True, existing auto-matched and rule-matched tags are removed
before applying the new set (inbox tags and manually-added tags are preserved).
Args:
document: The document to classify.
logging_group: Optional logging group for structured log output.
classifier: The trained classifier. If None, only rule-based matching runs.
replace: If True, remove existing classifier-managed tags before applying
new ones. Inbox tags and manually-added tags are always preserved.
dry_run: If True, compute what would change without saving anything.
**kwargs: Absorbed for Django signal compatibility (e.g. sender, signal).
Returns:
A two-tuple of (tags_added, tags_removed). In non-replace mode,
tags_removed is always an empty set. In dry_run mode, neither set
is applied to the database.
"""
# Compute which tags would be removed under replace mode.
# The filter mirrors the .delete() call below: keep inbox tags and
# manually-added tags (match="" and not auto-matched).
if replace:
tags_to_remove: set[Tag] = set(
document.tags.exclude(
is_inbox_tag=True,
).exclude(
Q(match="") & ~Q(matching_algorithm=Tag.MATCH_AUTO),
),
)
else:
tags_to_remove = set()
if replace and not dry_run:
Document.tags.through.objects.filter(document=document).exclude(
Q(tag__is_inbox_tag=True),
).exclude(
@@ -235,65 +261,53 @@ def set_tags(
).delete()
current_tags = set(document.tags.all())
matched_tags = matching.match_tags(document, classifier)
tags_to_add = set(matched_tags) - current_tags
relevant_tags = set(matched_tags) - current_tags
if suggest:
extra_tags = current_tags - set(matched_tags)
extra_tags = [
t for t in extra_tags if t.matching_algorithm == MatchingModel.MATCH_AUTO
]
if not relevant_tags and not extra_tags:
return
doc_str = style_func.SUCCESS(str(document))
if base_url:
stdout.write(doc_str)
stdout.write(f"{base_url}/documents/{document.pk}")
else:
stdout.write(doc_str + style_func.SUCCESS(f" [{document.pk}]"))
if relevant_tags:
stdout.write("Suggest tags: " + ", ".join([t.name for t in relevant_tags]))
if extra_tags:
stdout.write("Extra tags: " + ", ".join([t.name for t in extra_tags]))
else:
if not relevant_tags:
return
message = 'Tagging "{}" with "{}"'
if tags_to_add and not dry_run:
logger.info(
message.format(document, ", ".join([t.name for t in relevant_tags])),
f'Tagging "{document}" with "{", ".join(t.name for t in tags_to_add)}"',
extra={"group": logging_group},
)
document.add_nested_tags(tags_to_add)
document.add_nested_tags(relevant_tags)
return tags_to_add, tags_to_remove
def set_storage_path(
sender,
sender: object,
document: Document,
*,
logging_group=None,
logging_group: object = None,
classifier: DocumentClassifier | None = None,
replace=False,
use_first=True,
suggest=False,
base_url=None,
stdout=None,
style_func=None,
**kwargs,
) -> None:
replace: bool = False,
use_first: bool = True,
dry_run: bool = False,
**kwargs: Any,
) -> StoragePath | None:
"""
Assign a storage path to a document based on classifier results.
Args:
document: The document to classify.
logging_group: Optional logging group for structured log output.
classifier: The trained classifier. If None, only rule-based matching runs.
replace: If True, overwrite an existing storage path assignment.
use_first: If True, pick the first match when multiple paths match.
If False, skip assignment when multiple match.
dry_run: If True, compute and return the selection without saving.
**kwargs: Absorbed for Django signal compatibility (e.g. sender, signal).
Returns:
The storage path that was (or would be) assigned, or None if no match
was found or assignment was skipped.
"""
if document.storage_path and not replace:
return
return None
potential_storage_path = matching.match_storage_paths(
document,
classifier,
)
potential_count = len(potential_storage_path)
selected = potential_storage_path[0] if potential_storage_path else None
potential_storage_paths = matching.match_storage_paths(document, classifier)
potential_count = len(potential_storage_paths)
selected = potential_storage_paths[0] if potential_storage_paths else None
if potential_count > 1:
if use_first:
@@ -308,26 +322,17 @@ def set_storage_path(
f"not assigning any storage directory",
extra={"group": logging_group},
)
return
return None
if selected or replace:
if suggest:
_suggestion_printer(
stdout,
style_func,
"storage directory",
document,
selected,
base_url,
)
else:
logger.info(
f"Assigning storage path {selected} to {document}",
extra={"group": logging_group},
)
if (selected or replace) and not dry_run:
logger.info(
f"Assigning storage path {selected} to {document}",
extra={"group": logging_group},
)
document.storage_path = selected
document.save(update_fields=("storage_path",))
document.storage_path = selected
document.save(update_fields=("storage_path",))
return selected
# see empty_trash in documents/tasks.py for signal handling
@@ -467,8 +472,22 @@ def update_filename_and_move_files(
old_filename = instance.filename
old_source_path = instance.source_path
move_original = False
old_archive_filename = instance.archive_filename
old_archive_path = instance.archive_path
move_archive = False
candidate_filename = generate_filename(instance)
if len(str(candidate_filename)) > Document.MAX_STORED_FILENAME_LENGTH:
msg = (
f"Document {instance!s}: Generated filename exceeds db path "
f"limit ({len(str(candidate_filename))} > "
f"{Document.MAX_STORED_FILENAME_LENGTH}): {candidate_filename!s}"
)
logger.warning(msg)
raise CannotMoveFilesException(msg)
candidate_source_path = (
settings.ORIGINALS_DIR / candidate_filename
).resolve()
@@ -487,11 +506,16 @@ def update_filename_and_move_files(
instance.filename = str(new_filename)
move_original = old_filename != instance.filename
old_archive_filename = instance.archive_filename
old_archive_path = instance.archive_path
if instance.has_archive_version:
archive_candidate = generate_filename(instance, archive_filename=True)
if len(str(archive_candidate)) > Document.MAX_STORED_FILENAME_LENGTH:
msg = (
f"Document {instance!s}: Generated archive filename exceeds "
f"db path limit ({len(str(archive_candidate))} > "
f"{Document.MAX_STORED_FILENAME_LENGTH}): {archive_candidate!s}"
)
logger.warning(msg)
raise CannotMoveFilesException(msg)
archive_candidate_path = (
settings.ARCHIVE_DIR / archive_candidate
).resolve()
@@ -596,6 +620,16 @@ def update_filename_and_move_files(
root=settings.ARCHIVE_DIR,
)
# Keep version files in sync with root
if instance.root_document_id is None:
for version_doc in Document.objects.filter(root_document_id=instance.pk).only(
"pk",
):
update_filename_and_move_files(
Document,
version_doc,
)
@shared_task
def process_cf_select_update(custom_field: CustomField) -> None:
@@ -762,6 +796,28 @@ def run_workflows_updated(
)
def send_websocket_document_updated(
sender,
document: Document,
**kwargs,
) -> None:
# At this point, workflows may already have applied additional changes.
document.refresh_from_db()
from documents.data_models import DocumentMetadataOverrides
doc_overrides = DocumentMetadataOverrides.from_document(document)
with DocumentsStatusManager() as status_mgr:
status_mgr.send_document_updated(
document_id=document.id,
modified=DRF_DATETIME_FIELD.to_representation(document.modified),
owner_id=doc_overrides.owner_id,
users_can_view=doc_overrides.view_users,
groups_can_view=doc_overrides.view_groups,
)
def run_workflows(
trigger_type: WorkflowTrigger.WorkflowTriggerType,
document: Document | ConsumableDocument,
@@ -782,6 +838,14 @@ def run_workflows(
"""
use_overrides = overrides is not None
if isinstance(document, Document) and document.root_document_id is not None:
logger.debug(
"Skipping workflow execution for version document %s",
document.pk,
)
return None
if original_file is None:
original_file = (
document.source_path if not use_overrides else document.original_file
@@ -1035,7 +1099,11 @@ def add_or_update_document_in_llm_index(sender, document, **kwargs):
@receiver(models.signals.post_delete, sender=Document)
def delete_document_from_llm_index(sender, instance: Document, **kwargs):
def delete_document_from_llm_index(
sender: Any,
instance: Document,
**kwargs: Any,
) -> None:
"""
Delete a document from the LLM index when it is deleted.
"""

View File

@@ -4,11 +4,13 @@ import logging
import shutil
import uuid
import zipfile
from collections.abc import Callable
from collections.abc import Iterable
from pathlib import Path
from tempfile import TemporaryDirectory
from tempfile import mkstemp
from typing import TypeVar
import tqdm
from celery import Task
from celery import shared_task
from celery import states
@@ -60,17 +62,26 @@ from documents.sanity_checker import SanityCheckFailedException
from documents.signals import document_updated
from documents.signals.handlers import cleanup_document_deletion
from documents.signals.handlers import run_workflows
from documents.signals.handlers import send_websocket_document_updated
from documents.workflows.utils import get_workflows_for_trigger
from paperless.config import AIConfig
from paperless_ai.indexing import llm_index_add_or_update_document
from paperless_ai.indexing import llm_index_remove_document
from paperless_ai.indexing import update_llm_index
_T = TypeVar("_T")
IterWrapper = Callable[[Iterable[_T]], Iterable[_T]]
if settings.AUDIT_LOG_ENABLED:
from auditlog.models import LogEntry
logger = logging.getLogger("paperless.tasks")
def _identity(iterable: Iterable[_T]) -> Iterable[_T]:
return iterable
@shared_task
def index_optimize() -> None:
ix = index.open_index()
@@ -78,13 +89,13 @@ def index_optimize() -> None:
writer.commit(optimize=True)
def index_reindex(*, progress_bar_disable=False) -> None:
def index_reindex(*, iter_wrapper: IterWrapper[Document] = _identity) -> None:
documents = Document.objects.all()
ix = index.open_index(recreate=True)
with AsyncWriter(ix) as writer:
for document in tqdm.tqdm(documents, disable=progress_bar_disable):
for document in iter_wrapper(documents):
index.update_document(writer, document)
@@ -227,20 +238,30 @@ def consume_file(
@shared_task
def sanity_check(*, scheduled=True, raise_on_error=True):
messages = sanity_checker.check_sanity(scheduled=scheduled)
messages.log_messages()
if not messages.has_error and not messages.has_warning and not messages.has_info:
return "No issues detected."
parts: list[str] = []
if messages.document_error_count:
parts.append(f"{messages.document_error_count} document(s) with errors")
if messages.document_warning_count:
parts.append(f"{messages.document_warning_count} document(s) with warnings")
if messages.document_info_count:
parts.append(f"{messages.document_info_count} document(s) with infos")
if messages.global_warning_count:
parts.append(f"{messages.global_warning_count} global warning(s)")
summary = ", ".join(parts) + " found."
if messages.has_error:
message = "Sanity check exited with errors. See log."
message = summary + " Check logs for details."
if raise_on_error:
raise SanityCheckFailedException(message)
return message
elif messages.has_warning:
return "Sanity check exited with warnings. See log."
elif len(messages) > 0:
return "Sanity check exited with infos. See log."
else:
return "No issues detected."
return summary
@shared_task
@@ -265,7 +286,6 @@ def bulk_update_documents(document_ids) -> None:
ai_config = AIConfig()
if ai_config.llm_index_enabled:
update_llm_index(
progress_bar_disable=True,
rebuild=False,
)
@@ -452,13 +472,22 @@ def check_scheduled_workflows() -> None:
match trigger.schedule_date_field:
case WorkflowTrigger.ScheduleDateField.ADDED:
documents = Document.objects.filter(added__lte=threshold)
documents = Document.objects.filter(
root_document__isnull=True,
added__lte=threshold,
)
case WorkflowTrigger.ScheduleDateField.CREATED:
documents = Document.objects.filter(created__lte=threshold)
documents = Document.objects.filter(
root_document__isnull=True,
created__lte=threshold,
)
case WorkflowTrigger.ScheduleDateField.MODIFIED:
documents = Document.objects.filter(modified__lte=threshold)
documents = Document.objects.filter(
root_document__isnull=True,
modified__lte=threshold,
)
case WorkflowTrigger.ScheduleDateField.CUSTOM_FIELD:
# cap earliest date to avoid massive scans
@@ -496,7 +525,10 @@ def check_scheduled_workflows() -> None:
)
]
documents = Document.objects.filter(id__in=matched_ids)
documents = Document.objects.filter(
root_document__isnull=True,
id__in=matched_ids,
)
if documents.count() > 0:
documents = prefilter_documents_by_workflowtrigger(
@@ -541,6 +573,11 @@ def check_scheduled_workflows() -> None:
workflow_to_run=workflow,
document=document,
)
# Scheduled workflows dont send document_updated signal, so send a websocket update here to ensure clients are updated
send_websocket_document_updated(
sender=None,
document=document,
)
def update_document_parent_tags(tag: Tag, new_parent: Tag) -> None:
@@ -594,7 +631,7 @@ def update_document_parent_tags(tag: Tag, new_parent: Tag) -> None:
@shared_task
def llmindex_index(
*,
progress_bar_disable=True,
iter_wrapper: IterWrapper[Document] = _identity,
rebuild=False,
scheduled=True,
auto=False,
@@ -617,7 +654,7 @@ def llmindex_index(
try:
result = update_llm_index(
progress_bar_disable=progress_bar_disable,
iter_wrapper=iter_wrapper,
rebuild=rebuild,
)
task.status = states.SUCCESS

View File

@@ -79,6 +79,23 @@ class PlaceholderString(str):
NO_VALUE_PLACEHOLDER = PlaceholderString("-none-")
class MatchingModelContext:
"""
Safe template context for related objects.
Keeps legacy behavior where including the object ina template yields the related object's
name as a string, while still exposing limited attributes.
"""
def __init__(self, *, id: int, name: str, path: str | None = None):
self.id = id
self.name = name
self.path = path
def __str__(self) -> str:
return self.name
_template_environment.undefined = _LogStrictUndefined
_template_environment.filters["get_cf_value"] = get_cf_value
@@ -220,19 +237,26 @@ def get_safe_document_context(
else None,
"tags": [{"name": tag.name, "id": tag.id} for tag in tags],
"correspondent": (
{"name": document.correspondent.name, "id": document.correspondent.id}
MatchingModelContext(
name=document.correspondent.name,
id=document.correspondent.id,
)
if document.correspondent
else None
),
"document_type": (
{"name": document.document_type.name, "id": document.document_type.id}
MatchingModelContext(
name=document.document_type.name,
id=document.document_type.id,
)
if document.document_type
else None
),
"storage_path": {
"path": document.storage_path.path,
"id": document.storage_path.id,
}
"storage_path": MatchingModelContext(
name=document.storage_path.name,
path=document.storage_path.path,
id=document.storage_path.id,
)
if document.storage_path
else None,
}

Some files were not shown because too many files have changed in this diff Show More