Compare commits

..
Author SHA1 Message Date
stumpylog e85168a616 Mocks this test to pass on 3.14 too 2026-08-05 15:17:04 -07:00
stumpylogandClaude Sonnet 5 1e53db242d Fix: cover zstd-rejection path and correct importer's zstd-hint message
Adds a command-level test exercising the real zstd-unavailable branch
in document_exporter, makes document_importer only append the
"zstd archives require Python 3.14+" hint when zstd is actually among
the unreadable codecs, and adds the lzma counterpart to the
stored-level-rejection test for symmetry.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 14:45:44 -07:00
stumpylog f25a41919a Fix: narrow zstd compression level to the conventional -22..22 range
The raw library bounds (-131072, 22) come from an internal zstd
implementation constant (-ZSTD_TARGETLENGTH_MAX), not a meaningful
distinct level — deeper negative values than -22 buy nothing over -22
in practice, and the zstd CLI/community convention only uses -22..22.
Exposing the raw range via --zip-compression-level would let a user
pass a number like -50000 that "validates" but means nothing.
2026-08-05 14:45:44 -07:00
stumpylog f5a376537f Docs: document --zip-compression and --zip-compression-level 2026-08-05 14:45:44 -07:00
stumpylog 5904100135 Feature: importer rejects archives with unreadable compression 2026-08-05 14:45:44 -07:00
stumpylog 9dc5498de9 Test: add GIVEN/WHEN/THEN docstrings to compression tests
Matches the project's existing test docstring convention.
2026-08-05 14:45:44 -07:00
stumpylog f83eda193b Test: assert zip-compression flag resolves to the right sink constant
test_zip_lzma_compression_round_trips / test_default_zip_uses_deflate
built a real zip and read compress_type back off it — that only
reconfirms zipfile.ZipFile() honors its own compression= kwarg
(ZipExportSink's own tests already cover that forwarding). What this
command owns is resolving the --zip-compression string to the right
zipfile constant; assert that resolution directly against the mocked
ZipExportSink construction call instead. Drops the sample-doc copytree
setup those tests needed only to build a real archive.
2026-08-05 14:45:44 -07:00
stumpylog df37eecfda Test: assert compression forwarding via call args, not a real archive
test_compression_method_is_applied_to_file_entries built a real zip and
read back compress_type — but that only confirms zipfile.ZipFile()
honors its own compression= kwarg, which is documented stdlib behavior,
not something our code could get wrong. ZipExportSink's only
responsibility is forwarding compression/compresslevel unchanged to
ZipFile(); assert that directly against the mocked constructor call.
2026-08-05 14:45:43 -07:00
stumpylog dfe6a41618 Test: remove test asserting Python's own compression behavior
test_compressing_method_beats_stored asserted that DEFLATED produces a
smaller archive than STORED — that's zlib's job, not ours.
test_compression_method_is_applied_to_file_entries already covers what
our code is actually responsible for: threading the requested
compression method through to the zip entry's compress_type.
2026-08-05 14:43:25 -07:00
stumpylog e85be4f18e Feature: add --zip-compression and --zip-compression-level flags 2026-08-05 14:43:24 -07:00
stumpylog b9251fe6ee Feature: ZipExportSink accepts compression method and level 2026-08-05 14:43:24 -07:00
stumpylog 423833f82c Feature: add export compression policy module 2026-08-05 14:43:24 -07:00
Trenton HandGitHub 731b3403d2 Merge branch 'dev' into feature-direct-zip-export 2026-08-05 13:57:35 -07:00
stumpylog d381cf74d5 Sure, defense in depth against odd things 2026-08-05 13:21:45 -07:00
Trenton Holmes 1f2de612ce Increase test coverage 2026-08-05 13:21:45 -07:00
Trenton Holmes d40af75461 Refactor: stream documents via QuerySetStream in exporter instead of eager dict 2026-08-05 13:21:45 -07:00
stumpylog 3f2a4dd9eb Polish: atomic zip commit via Path.replace, widen sink params to ExportSink
Path.rename() raises FileExistsError on Windows when the destination
already exists; Path.replace() is atomic cross-platform. Also widen
document_exporter's sink parameters from the concrete
DirectoryExportSink | ZipExportSink union to the ExportSink ABC, so a
future sink implementation is a pure addition rather than requiring
every call site's annotation to change.
2026-08-05 13:21:45 -07:00
stumpylog 71f5a94bae Refactor: de-duplicate BLAKE2b compare and simplify zip dir-marker loop
DirectoryExportSink.add_json and _commit_streamed_file each inlined the
same hash-and-compare logic; extracted _content_unchanged(). Replaced
ZipExportSink._ensure_dirs's repeated slice/join with a prefix
accumulator and hoisted the _zip-is-open assertion out of the loop. No
behavioral change.
2026-08-05 13:21:45 -07:00
stumpylog 4719eeac9a Fix: annotate ExportSink.stream's return type for pyrefly
The abstract method had no return annotation, so pyrefly inferred -> None
and flagged both DirectoryExportSink.stream and ZipExportSink.stream as
incompatible overrides.
2026-08-05 13:21:44 -07:00
stumpylog 6d5da5d92f Test: guard --zip combined with --compare-* flags 2026-08-05 13:21:44 -07:00
stumpylogandClaude Sonnet 5 62a6b1835e Refactor: route document_exporter through ExportSink, direct-to-zip
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-05 13:21:44 -07:00
stumpylog 1bf9140345 Fix: de-duplicate source_file fixture across sink test classes
TestDirectoryExportSink and TestZipExportSink each defined an
identical source_file fixture; hoist it to module scope.
2026-08-05 13:21:44 -07:00
stumpylog d592bd19e1 Feature: add ZipExportSink with atomic finalize and manifest spooling 2026-08-05 13:21:44 -07:00
stumpylog a8ddeecb77 Fix: make ExportSink a real ABC per design spec
The implementation plan for Task 2 diverged from the design spec
(export-sink-architecture-design.md), leaving ExportSink as a plain
class with NotImplementedError bodies instead of the specified
AbstractContextManager subclass. Use abc.ABC + @abstractmethod so a
concrete sink missing a required method fails at instantiation
rather than at first call.
2026-08-05 13:21:44 -07:00
stumpylog f76ac48d69 Feature: add ExportSink ABC and DirectoryExportSink 2026-08-05 13:21:44 -07:00
stumpylog c5c8aecb73 Feature: add export package with StreamingManifestWriter and _dumps 2026-08-05 13:21:44 -07:00
41 changed files with 4009 additions and 3838 deletions
+19 -14
View File
@@ -129,8 +129,8 @@ jobs:
~/.pnpm-store ~/.pnpm-store
~/.cache ~/.cache
key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }} key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
- name: Install dependencies - name: Re-link Angular CLI
run: cd src-ui && pnpm install --frozen-lockfile run: cd src-ui && pnpm link @angular/cli
- name: Run lint - name: Run lint
run: cd src-ui && pnpm run lint run: cd src-ui && pnpm run lint
unit-tests: unit-tests:
@@ -168,8 +168,8 @@ jobs:
~/.pnpm-store ~/.pnpm-store
~/.cache ~/.cache
key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }} key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
- name: Install dependencies - name: Re-link Angular CLI
run: cd src-ui && pnpm install --frozen-lockfile run: cd src-ui && pnpm link @angular/cli
- name: Run Jest unit tests - name: Run Jest unit tests
run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }} run: cd src-ui && pnpm run test --max-workers=2 --shard=${{ matrix.shard-index }}/${{ matrix.shard-count }}
- name: Upload test results to Codecov - name: Upload test results to Codecov
@@ -223,15 +223,18 @@ jobs:
~/.pnpm-store ~/.pnpm-store
~/.cache ~/.cache
key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }} key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
- name: Re-link Angular CLI
run: cd src-ui && pnpm link @angular/cli
- name: Install dependencies - name: Install dependencies
run: cd src-ui && pnpm install --frozen-lockfile run: cd src-ui && pnpm install --no-frozen-lockfile
- name: Run Playwright E2E tests - name: Run Playwright E2E tests
run: cd src-ui && pnpm exec playwright test --shard ${{ matrix.shard-index }}/${{ matrix.shard-count }} run: cd src-ui && pnpm exec playwright test --shard ${{ matrix.shard-index }}/${{ matrix.shard-count }}
frontend-build: bundle-analysis:
name: Frontend Build name: Bundle Analysis
needs: [changes, unit-tests, e2e-tests] needs: [changes, unit-tests, e2e-tests]
if: needs.changes.outputs.frontend_changed == 'true' if: needs.changes.outputs.frontend_changed == 'true'
runs-on: ubuntu-24.04 runs-on: ubuntu-24.04
environment: bundle-analysis
permissions: permissions:
contents: read contents: read
steps: steps:
@@ -257,19 +260,21 @@ jobs:
~/.pnpm-store ~/.pnpm-store
~/.cache ~/.cache
key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }} key: ${{ runner.os }}-frontend-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
- name: Install dependencies - name: Re-link Angular CLI
run: cd src-ui && pnpm install --frozen-lockfile run: cd src-ui && pnpm link @angular/cli
- name: Build - name: Build and analyze
env:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
run: cd src-ui && pnpm run build --configuration=production run: cd src-ui && pnpm run build --configuration=production
gate: gate:
name: Frontend CI Gate name: Frontend CI Gate
needs: [changes, install-dependencies, lint, unit-tests, e2e-tests, frontend-build] needs: [changes, install-dependencies, lint, unit-tests, e2e-tests, bundle-analysis]
if: always() if: always()
runs-on: ubuntu-slim runs-on: ubuntu-slim
steps: steps:
- name: Check gate - name: Check gate
env: env:
BUILD_RESULT: ${{ needs['frontend-build'].result }} BUNDLE_ANALYSIS_RESULT: ${{ needs['bundle-analysis'].result }}
E2E_RESULT: ${{ needs['e2e-tests'].result }} E2E_RESULT: ${{ needs['e2e-tests'].result }}
FRONTEND_CHANGED: ${{ needs.changes.outputs.frontend_changed }} FRONTEND_CHANGED: ${{ needs.changes.outputs.frontend_changed }}
INSTALL_RESULT: ${{ needs['install-dependencies'].result }} INSTALL_RESULT: ${{ needs['install-dependencies'].result }}
@@ -301,8 +306,8 @@ jobs:
exit 1 exit 1
fi fi
if [[ "${BUILD_RESULT}" != "success" ]]; then if [[ "${BUNDLE_ANALYSIS_RESULT}" != "success" ]]; then
echo "::error::Frontend build job result: ${BUILD_RESULT}" echo "::error::Frontend bundle-analysis job result: ${BUNDLE_ANALYSIS_RESULT}"
exit 1 exit 1
fi fi
+4 -1
View File
@@ -61,7 +61,10 @@ jobs:
~/.cache ~/.cache
key: ${{ runner.os }}-frontenddeps-${{ hashFiles('src-ui/pnpm-lock.yaml') }} key: ${{ runner.os }}-frontenddeps-${{ hashFiles('src-ui/pnpm-lock.yaml') }}
- name: Install frontend dependencies - name: Install frontend dependencies
run: cd src-ui && pnpm install --frozen-lockfile if: steps.cache-frontend-deps.outputs.cache-hit != 'true'
run: cd src-ui && pnpm install
- name: Re-link Angular cli
run: cd src-ui && pnpm link @angular/cli
- name: Generate frontend translation strings - name: Generate frontend translation strings
run: | run: |
cd src-ui cd src-ui
+15
View File
@@ -299,6 +299,8 @@ optional arguments:
-sm, --split-manifest -sm, --split-manifest
-z, --zip -z, --zip
-zn, --zip-name -zn, --zip-name
--zip-compression
--zip-compression-level
--data-only --data-only
--no-progress-bar --no-progress-bar
--passphrase --passphrase
@@ -361,6 +363,19 @@ If `-z` or `--zip` is provided, the export will be a zip file
in the target directory, named according to the current local date or the in the target directory, named according to the current local date or the
value set in `-zn` or `--zip-name`. value set in `-zn` or `--zip-name`.
The compression method for the zip can be set with `--zip-compression`
(`stored`, `deflated` (default), `bzip2`, `lzma`, or `zstd`) and tuned with
`--zip-compression-level` (deflated: 09, bzip2: 19, zstd: -2222; ignored
for `stored` and `lzma`). Both options require `--zip`.
!!! warning
`zstd` compression requires Python 3.14 or newer on **both** the machine
creating the export and any machine importing it. An archive compressed with
`zstd` (or `lzma`/`bzip2` where those modules are unavailable) cannot be
imported on a runtime that lacks the codec; the importer will refuse it with
a clear error. The default `deflated` is universally readable.
If `--data-only` is provided, only the database will be exported. This option is intended If `--data-only` is provided, only the database will be exported. This option is intended
to facilitate database upgrades without needing to clean documents and thumbnails from the media directory. to facilitate database upgrades without needing to clean documents and thumbnails from the media directory.
+1
View File
@@ -38,6 +38,7 @@ dependencies = [
"django-soft-delete~=1.0.18", "django-soft-delete~=1.0.18",
"django-treenode>=0.24", "django-treenode>=0.24",
"djangorestframework~=3.16", "djangorestframework~=3.16",
"djangorestframework-guardian~=0.4.0",
"drf-spectacular~=0.30", "drf-spectacular~=0.30",
"drf-spectacular-sidecar~=2026.7.1", "drf-spectacular-sidecar~=2026.7.1",
"drf-writable-nested~=0.7.1", "drf-writable-nested~=0.7.1",
+9 -12
View File
@@ -56,13 +56,13 @@
}, },
"architect": { "architect": {
"build": { "build": {
"builder": "@angular/build:application", "builder": "@angular-builders/custom-webpack:browser",
"options": { "options": {
"outputPath": { "customWebpackConfig": {
"base": "dist/paperless-ui", "path": "./extra-webpack.config.ts"
"browser": ""
}, },
"browser": "src/main.ts", "outputPath": "dist/paperless-ui",
"main": "src/main.ts",
"outputHashing": "none", "outputHashing": "none",
"index": "src/index.html", "index": "src/index.html",
"polyfills": [ "polyfills": [
@@ -97,7 +97,6 @@
"scripts": [], "scripts": [],
"allowedCommonJsDependencies": [ "allowedCommonJsDependencies": [
"file-saver", "file-saver",
"mime-names",
"utif" "utif"
], ],
"extractLicenses": false, "extractLicenses": false,
@@ -118,13 +117,11 @@
"with": "src/environments/environment.prod.ts" "with": "src/environments/environment.prod.ts"
} }
], ],
"outputPath": { "outputPath": "../src/documents/static/frontend/",
"base": "../src/documents/static/frontend/",
"browser": ""
},
"optimization": true, "optimization": true,
"outputHashing": "none", "outputHashing": "none",
"sourceMap": false, "sourceMap": false,
"namedChunks": false,
"extractLicenses": true, "extractLicenses": true,
"budgets": [ "budgets": [
{ {
@@ -148,7 +145,7 @@
"defaultConfiguration": "" "defaultConfiguration": ""
}, },
"serve": { "serve": {
"builder": "@angular/build:dev-server", "builder": "@angular-builders/custom-webpack:dev-server",
"options": { "options": {
"buildTarget": "paperless-ui:build:en-US" "buildTarget": "paperless-ui:build:en-US"
}, },
@@ -159,7 +156,7 @@
} }
}, },
"extract-i18n": { "extract-i18n": {
"builder": "@angular/build:extract-i18n", "builder": "@angular-builders/custom-webpack:extract-i18n",
"options": { "options": {
"buildTarget": "paperless-ui:build" "buildTarget": "paperless-ui:build"
} }
+24
View File
@@ -0,0 +1,24 @@
import {
CustomWebpackBrowserSchema,
TargetOptions,
} from '@angular-builders/custom-webpack'
import * as webpack from 'webpack'
const { codecovWebpackPlugin } = require('@codecov/webpack-plugin')
export default (
config: webpack.Configuration,
options: CustomWebpackBrowserSchema,
targetOptions: TargetOptions
) => {
if (config.plugins) {
config.plugins.push(
codecovWebpackPlugin({
enableBundleAnalysis: process.env.CODECOV_TOKEN !== undefined,
bundleName: 'paperless-ngx',
uploadToken: process.env.CODECOV_TOKEN,
})
)
}
return config
}
+711 -710
View File
File diff suppressed because it is too large Load Diff
+17 -14
View File
@@ -12,13 +12,13 @@
"private": true, "private": true,
"dependencies": { "dependencies": {
"@angular/cdk": "^22.0.6", "@angular/cdk": "^22.0.6",
"@angular/common": "~22.1.0", "@angular/common": "~22.0.8",
"@angular/compiler": "~22.1.0", "@angular/compiler": "~22.0.8",
"@angular/core": "~22.1.0", "@angular/core": "~22.0.8",
"@angular/forms": "~22.1.0", "@angular/forms": "~22.0.8",
"@angular/localize": "~22.1.0", "@angular/localize": "~22.0.8",
"@angular/platform-browser": "~22.1.0", "@angular/platform-browser": "~22.0.8",
"@angular/router": "~22.1.0", "@angular/router": "~22.0.8",
"@ng-bootstrap/ng-bootstrap": "^21.0.0", "@ng-bootstrap/ng-bootstrap": "^21.0.0",
"@ng-select/ng-select": "^23.5.0", "@ng-select/ng-select": "^23.5.0",
"@ngneat/dirty-check-forms": "^3.0.3", "@ngneat/dirty-check-forms": "^3.0.3",
@@ -32,24 +32,26 @@
"ngx-device-detector": "^12.0.0", "ngx-device-detector": "^12.0.0",
"ngx-ui-tour-ng-bootstrap": "^19.0.0", "ngx-ui-tour-ng-bootstrap": "^19.0.0",
"normalize-diacritics": "^5.0.0", "normalize-diacritics": "^5.0.0",
"pdfjs-dist": "^6.2.108", "pdfjs-dist": "^6.0.227",
"rxjs": "^7.8.2", "rxjs": "^7.8.2",
"tslib": "^2.8.1", "tslib": "^2.8.1",
"utif": "^3.1.0", "utif": "^3.1.0",
"uuid": "^14.0.1" "uuid": "^14.0.1"
}, },
"devDependencies": { "devDependencies": {
"@angular-builders/custom-webpack": "^22.0.1",
"@angular-builders/jest": "^22.0.1", "@angular-builders/jest": "^22.0.1",
"@angular-devkit/core": "^22.1.2", "@angular-devkit/core": "^22.0.8",
"@angular-devkit/schematics": "^22.1.2", "@angular-devkit/schematics": "^22.0.8",
"@angular-eslint/builder": "22.1.0", "@angular-eslint/builder": "22.1.0",
"@angular-eslint/eslint-plugin": "22.1.0", "@angular-eslint/eslint-plugin": "22.1.0",
"@angular-eslint/eslint-plugin-template": "22.1.0", "@angular-eslint/eslint-plugin-template": "22.1.0",
"@angular-eslint/schematics": "22.1.0", "@angular-eslint/schematics": "22.1.0",
"@angular-eslint/template-parser": "22.1.0", "@angular-eslint/template-parser": "22.1.0",
"@angular/build": "22.1.2", "@angular/build": "^22.0.8",
"@angular/cli": "22.1.2", "@angular/cli": "~22.0.5",
"@angular/compiler-cli": "~22.1.0", "@angular/compiler-cli": "~22.0.8",
"@codecov/webpack-plugin": "^2.0.1",
"@playwright/test": "^1.62.0", "@playwright/test": "^1.62.0",
"@types/jest": "^30.0.0", "@types/jest": "^30.0.0",
"@types/node": "^26.1.1", "@types/node": "^26.1.1",
@@ -64,7 +66,8 @@
"jest-websocket-mock": "^2.5.0", "jest-websocket-mock": "^2.5.0",
"prettier-plugin-organize-imports": "^4.3.0", "prettier-plugin-organize-imports": "^4.3.0",
"ts-node": "~10.9.1", "ts-node": "~10.9.1",
"typescript": "^6.0.3" "typescript": "^6.0.3",
"webpack": "^5.107.2"
}, },
"packageManager": "pnpm@10.26.0" "packageManager": "pnpm@10.26.0"
} }
+1798 -1811
View File
File diff suppressed because it is too large Load Diff
@@ -151,13 +151,6 @@
inset: 0; inset: 0;
pointer-events: none; pointer-events: none;
& section {
position: absolute;
text-align: initial;
box-sizing: border-box;
transform-origin: 0 0;
}
& .annotationTextContent { & .annotationTextContent {
opacity: 0; opacity: 0;
} }
@@ -13,7 +13,6 @@ import {
ViewChild, ViewChild,
} from '@angular/core' } from '@angular/core'
import { import {
AnnotationMode,
getDocument, getDocument,
GlobalWorkerOptions, GlobalWorkerOptions,
PDFDocumentLoadingTask, PDFDocumentLoadingTask,
@@ -222,7 +221,6 @@ export class PngxPdfViewerComponent
linkService: this.linkService, linkService: this.linkService,
findController: this.findController, findController: this.findController,
textLayerMode, textLayerMode,
annotationMode: AnnotationMode.ENABLE,
enableSelectionRendering: false, enableSelectionRendering: false,
removePageBorders: true, removePageBorders: true,
} }
@@ -2161,14 +2161,8 @@ describe('DocumentDetailComponent', () => {
it('should support open share links and email modals', () => { it('should support open share links and email modals', () => {
const modalSpy = jest.spyOn(modalService, 'open') const modalSpy = jest.spyOn(modalService, 'open')
initNormally() initNormally()
component.selectedVersionId.set(10)
component.openShareLinks() component.openShareLinks()
expect(modalSpy).toHaveBeenCalled() expect(modalSpy).toHaveBeenCalled()
expect(
(
modalSpy.mock.results[0].value as NgbModalRef
).componentInstance.documentId()
).toBe(10)
component.openEmailDocument() component.openEmailDocument()
expect(modalSpy).toHaveBeenCalled() expect(modalSpy).toHaveBeenCalled()
}) })
@@ -1959,9 +1959,7 @@ export class DocumentDetailComponent
public openShareLinks() { public openShareLinks() {
const modal = this.modalService.open(ShareLinksDialogComponent) const modal = this.modalService.open(ShareLinksDialogComponent)
modal.componentInstance.documentId.set( modal.componentInstance.documentId.set(this.document().id)
this.selectedVersionId() ?? this.document().id
)
modal.componentInstance.hasArchiveVersion.set( modal.componentInstance.hasArchiveVersion.set(
this.metadata()?.has_archive_version ?? this.metadata()?.has_archive_version ??
!!this.document()?.archived_file_name !!this.document()?.archived_file_name
@@ -2213,20 +2213,6 @@ describe('FilterEditorComponent', () => {
expect(blurSpy).toHaveBeenCalled() expect(blurSpy).toHaveBeenCalled()
}) })
it('should only dismiss open autocomplete suggestions on Escape, keeping the query', () => {
component.textFilter = 'foo bar'
component.textFilterInput.nativeElement.value = 'foo bar'
jest.spyOn(component.searchTypeahead, 'isPopupOpen').mockReturnValue(true)
const dismissSpy = jest
.spyOn(component.searchTypeahead, 'dismissPopup')
.mockImplementation(() => {})
component.textFilterInput.nativeElement.dispatchEvent(
new KeyboardEvent('keydown', { key: 'Escape' })
)
expect(dismissSpy).toHaveBeenCalled()
expect(component.textFilter).toEqual('foo bar')
})
it('should adjust text filter targets if more like search', () => { it('should adjust text filter targets if more like search', () => {
const TEXT_FILTER_TARGET_FULLTEXT_MORELIKE = 'fulltext-morelike' // private const const TEXT_FILTER_TARGET_FULLTEXT_MORELIKE = 'fulltext-morelike' // private const
component.textFilterTarget = TEXT_FILTER_TARGET_FULLTEXT_MORELIKE component.textFilterTarget = TEXT_FILTER_TARGET_FULLTEXT_MORELIKE
@@ -15,7 +15,6 @@ import {
import { FormsModule, ReactiveFormsModule } from '@angular/forms' import { FormsModule, ReactiveFormsModule } from '@angular/forms'
import { import {
NgbDropdownModule, NgbDropdownModule,
NgbTypeahead,
NgbTypeaheadModule, NgbTypeaheadModule,
} from '@ng-bootstrap/ng-bootstrap' } from '@ng-bootstrap/ng-bootstrap'
import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons' import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
@@ -352,9 +351,6 @@ export class FilterEditorComponent
@ViewChild('textFilterInput') @ViewChild('textFilterInput')
textFilterInput: ElementRef textFilterInput: ElementRef
@ViewChild(NgbTypeahead)
searchTypeahead: NgbTypeahead
readonly customFields = signal<CustomField[]>([]) readonly customFields = signal<CustomField[]>([])
tagDocumentCounts: SelectionDataItem[] tagDocumentCounts: SelectionDataItem[]
@@ -1154,7 +1150,6 @@ export class FilterEditorComponent
} }
set textFilter(value) { set textFilter(value) {
this._textFilter = value // set immediately to prevent loss of keystrokes
this.textFilterDebounce.next(value) this.textFilterDebounce.next(value)
} }
@@ -1247,9 +1242,9 @@ export class FilterEditorComponent
distinctUntilChanged(), distinctUntilChanged(),
filter((query) => !query.length || query.length > 2) filter((query) => !query.length || query.length > 2)
) )
.subscribe(() => .subscribe((text) =>
this.updateTextFilter( this.updateTextFilter(
this._textFilter, // use the current value, not the debounced (possibly stale) one text,
this.textFilterTarget !== TEXT_FILTER_TARGET_FULLTEXT_QUERY this.textFilterTarget !== TEXT_FILTER_TARGET_FULLTEXT_QUERY
) )
) )
@@ -1325,11 +1320,6 @@ export class FilterEditorComponent
this.updateTextFilter(filterString) this.updateTextFilter(filterString)
} }
} else if (event.key === 'Escape') { } else if (event.key === 'Escape') {
if (this.searchTypeahead?.isPopupOpen()) {
// only dismiss the suggestions, so longer query can use Enter
this.searchTypeahead.dismissPopup()
return
}
if (this._textFilter?.length) { if (this._textFilter?.length) {
this.resetTextField() this.resetTextField()
} else { } else {
@@ -88,7 +88,7 @@
@if (depth > 0) { @if (depth > 0) {
<div class="indicator"></div> <div class="indicator"></div>
} }
<button class="btn btn-link ms-0 ps-0 text-start" style="user-select: text;" [disabled]="!userCanEdit(object)" (click)="userCanEdit(object) ? openEditDialog(object) : null; $event.stopPropagation()">{{ object.name }}</button> <button class="btn btn-link ms-0 ps-0 text-start" (click)="userCanEdit(object) ? openEditDialog(object) : null; $event.stopPropagation()">{{ object.name }}</button>
</td> </td>
<td class="d-none d-sm-table-cell">{{ getMatching(object) }}</td> <td class="d-none d-sm-table-cell">{{ getMatching(object) }}</td>
<td>{{ getDocumentCount(object) }}</td> <td>{{ getDocumentCount(object) }}</td>
@@ -19,13 +19,6 @@ export const GlobalWorkerOptions = {
workerSrc: '', workerSrc: '',
} }
export const AnnotationMode = {
DISABLE: 0,
ENABLE: 1,
ENABLE_FORMS: 2,
ENABLE_STORAGE: 3,
}
export const getDocument = (_src: unknown): PDFDocumentLoadingTask => { export const getDocument = (_src: unknown): PDFDocumentLoadingTask => {
return new PDFDocumentLoadingTask(Promise.resolve(new PDFDocumentProxy())) return new PDFDocumentLoadingTask(Promise.resolve(new PDFDocumentProxy()))
} }
+108
View File
@@ -0,0 +1,108 @@
from __future__ import annotations
import importlib
import zipfile
# ZIP_ZSTANDARD exists only on Python 3.14+ (PEP 784). None elsewhere.
ZSTD: int | None = getattr(zipfile, "ZIP_ZSTANDARD", None)
# CLI choices are fixed across runtimes so argparse never hides zstd; runtime
# availability is enforced separately in compression_available().
COMPRESSION_CHOICES: tuple[str, ...] = (
"stored",
"deflated",
"bzip2",
"lzma",
"zstd",
)
# Method name -> zipfile compression constant (zstd only when supported).
COMPRESSION_METHODS: dict[str, int] = {
"stored": zipfile.ZIP_STORED,
"deflated": zipfile.ZIP_DEFLATED,
"bzip2": zipfile.ZIP_BZIP2,
"lzma": zipfile.ZIP_LZMA,
}
if ZSTD is not None:
COMPRESSION_METHODS["zstd"] = ZSTD
# Inclusive (min, max) level bounds per method; None => level not applicable.
# Verified on CPython 3.14.3.
#
# zstd's raw library bounds are (-131072, 22)
# (compression.zstd.CompressionParameter.compression_level.bounds()) — the
# minimum is an internal implementation constant (-ZSTD_TARGETLENGTH_MAX),
# not a meaningful distinct "level"; deeper negative values than -22 buy
# nothing over -22 in practice. We expose the conventional zstd CLI range
# instead of the raw library bounds.
LEVEL_BOUNDS: dict[str, tuple[int, int] | None] = {
"stored": None,
"deflated": (0, 9),
"bzip2": (1, 9),
"lzma": None,
"zstd": (-22, 22),
}
# zipfile compress_type id -> method name. 93 = current zstd id, 20 = legacy
# zstd id that zipfile can still read.
_COMPRESS_TYPE_TO_METHOD: dict[int, str] = {
zipfile.ZIP_STORED: "stored",
zipfile.ZIP_DEFLATED: "deflated",
zipfile.ZIP_BZIP2: "bzip2",
zipfile.ZIP_LZMA: "lzma",
93: "zstd",
20: "zstd",
}
def compression_available(method: str) -> bool:
"""Whether the running interpreter can actually use the given method."""
if method in ("stored", "deflated"):
# zlib is a hard CPython dependency; stored needs nothing.
return True
if method == "bzip2":
return _module_importable("bz2")
if method == "lzma":
return _module_importable("lzma")
if method == "zstd":
return ZSTD is not None and _module_importable("compression.zstd")
return False
def _module_importable(name: str) -> bool:
try:
importlib.import_module(name)
except ImportError:
return False
return True
def level_error(method: str, level: int | None) -> str | None:
"""Return a human message if (method, level) is invalid, else None."""
if level is None:
return None
bounds = LEVEL_BOUNDS[method]
if bounds is None:
return f"--zip-compression-level has no effect for '{method}'"
low, high = bounds
if not (low <= level <= high):
return (
f"--zip-compression-level for '{method}' must be between {low} and {high}"
)
return None
def compress_type_readable(compress_type: int) -> bool:
"""Whether this interpreter can decompress an entry of the given type."""
method = _COMPRESS_TYPE_TO_METHOD.get(compress_type)
if method is None:
return False
return compression_available(method)
def unreadable_method_names(compress_types: set[int]) -> set[str]:
"""Map a set of compress_type ids to human method names for error messages."""
names: set[str] = set()
for ct in compress_types:
names.add(_COMPRESS_TYPE_TO_METHOD.get(ct, f"method {ct}"))
return names
+13 -2
View File
@@ -243,11 +243,21 @@ class ZipExportSink(ExportSink):
added as an entry at finalize (a zip entry cannot be interleaved with others). added as an entry at finalize (a zip entry cannot be interleaved with others).
""" """
def __init__(self, target: Path, zip_name: str, *, delete: bool = False) -> None: def __init__(
self,
target: Path,
zip_name: str,
*,
delete: bool = False,
compression: int = zipfile.ZIP_DEFLATED,
compresslevel: int | None = None,
) -> None:
self._target = target.resolve() self._target = target.resolve()
self._zip_path = (self._target / zip_name).with_suffix(".zip") self._zip_path = (self._target / zip_name).with_suffix(".zip")
self._tmp_path = self._zip_path.with_name(self._zip_path.name + ".tmp") self._tmp_path = self._zip_path.with_name(self._zip_path.name + ".tmp")
self._delete = delete self._delete = delete
self._compression = compression
self._compresslevel = compresslevel
self._zip: zipfile.ZipFile | None = None self._zip: zipfile.ZipFile | None = None
self._dirs: set[str] = set() self._dirs: set[str] = set()
self._pending_manifest: tuple[Path, str] | None = None self._pending_manifest: tuple[Path, str] | None = None
@@ -258,7 +268,8 @@ class ZipExportSink(ExportSink):
self._zip = zipfile.ZipFile( self._zip = zipfile.ZipFile(
self._tmp_path, self._tmp_path,
"w", "w",
compression=zipfile.ZIP_DEFLATED, compression=self._compression,
compresslevel=self._compresslevel,
allowZip64=True, allowZip64=True,
) )
+49 -24
View File
@@ -39,6 +39,7 @@ from guardian.utils import get_user_obj_perms_model
from rest_framework import serializers from rest_framework import serializers
from rest_framework.filters import BaseFilterBackend from rest_framework.filters import BaseFilterBackend
from rest_framework.filters import OrderingFilter from rest_framework.filters import OrderingFilter
from rest_framework_guardian.filters import ObjectPermissionsFilter
from documents.models import Correspondent from documents.models import Correspondent
from documents.models import CustomField from documents.models import CustomField
@@ -50,7 +51,7 @@ from documents.models import ShareLink
from documents.models import ShareLinkBundle from documents.models import ShareLinkBundle
from documents.models import StoragePath from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from documents.permissions import permitted_object_ids from documents.permissions import permitted_document_ids
if TYPE_CHECKING: if TYPE_CHECKING:
from collections.abc import Callable from collections.abc import Callable
@@ -1027,35 +1028,59 @@ class PaperlessTaskFilterSet(FilterSet):
return queryset.exclude(status__in=PaperlessTask.COMPLETE_STATUSES) return queryset.exclude(status__in=PaperlessTask.COMPLETE_STATUSES)
class PermittedObjectsFilter(BaseFilterBackend): class ObjectOwnedOrGrantedPermissionsFilter(ObjectPermissionsFilter):
""" """
Filters a queryset down to objects the requesting user owns, are A filter backend that limits results to those where the requesting user
unowned, or (when ``include_granted`` is True) has an explicit has read object level permissions, owns the objects, or objects without
user/group guardian permission on. Backed by ``permitted_object_ids`` an owner (for backwards compat)
-- a single ``id__in`` subquery, not a join -- so it can't produce
duplicate rows even when the base queryset already carries independent
joins (e.g. multi-value ``tags__id__all`` filtering), and stays
index-friendly at scale instead of falling back to guardian's
varchar-cast join.
Set ``include_granted = False`` on a subclass for endpoints that
intentionally only show owned/unowned objects regardless of explicit
shares (e.g. ``TrashView``).
""" """
include_granted: bool = True
perm_codename: str | None = None
def filter_queryset(self, request, queryset, view): def filter_queryset(self, request, queryset, view):
if request.user.is_superuser: if request.user.is_superuser:
return queryset return queryset
if not self.include_granted: objects_with_perms = super().filter_queryset(request, queryset, view)
return queryset.filter(Q(owner=request.user) | Q(owner__isnull=True)) objects_owned = queryset.filter(owner=request.user)
model = queryset.model objects_unowned = queryset.filter(owner__isnull=True)
perm = self.perm_codename or f"view_{model._meta.model_name}" return objects_with_perms | objects_owned | objects_unowned
return queryset.filter(
id__in=permitted_object_ids(request.user, model, perm),
) class DocumentPermissionsFilter(BaseFilterBackend):
"""
A filter backend limiting Document results to those the requesting user
owns, are unowned, or has explicit (user- or group-level) view
permission on.
Unlike ``ObjectOwnedOrGrantedPermissionsFilter``, this does not build an
``objects_with_perms | objects_owned | objects_unowned`` union of
querysets derived from the same base queryset. When that base queryset
already carries independent joins on a multi-valued relation (e.g. two
separate joins from ``tags__id__all`` filtering on two tags), each
OR-ed branch can end up pairing those joins' aliases differently,
letting more than one row out of the join's cross product satisfy the
combined WHERE -- returning the same document more than once. Filtering
via a single ``id__in`` against ``permitted_document_ids`` (a plain
subquery, not a join) sidesteps that entirely and is also cheaper than
guardian's join-based permission check.
"""
def filter_queryset(self, request, queryset, view):
if request.user.is_superuser:
return queryset
return queryset.filter(id__in=permitted_document_ids(request.user))
class ObjectOwnedPermissionsFilter(ObjectPermissionsFilter):
"""
A filter backend that limits results to those where the requesting user
owns the objects or objects without an owner (for backwards compat)
"""
def filter_queryset(self, request, queryset, view):
if request.user.is_superuser:
return queryset
objects_owned = queryset.filter(owner=request.user)
objects_unowned = queryset.filter(owner__isnull=True)
return objects_owned | objects_unowned
class DocumentsOrderingFilter(OrderingFilter): class DocumentsOrderingFilter(OrderingFilter):
@@ -29,6 +29,11 @@ if TYPE_CHECKING:
if settings.AUDIT_LOG_ENABLED: if settings.AUDIT_LOG_ENABLED:
from auditlog.models import LogEntry from auditlog.models import LogEntry
from documents.export.compression import COMPRESSION_CHOICES
from documents.export.compression import COMPRESSION_METHODS
from documents.export.compression import ZSTD
from documents.export.compression import compression_available
from documents.export.compression import level_error
from documents.export.sinks import DirectoryExportSink from documents.export.sinks import DirectoryExportSink
from documents.export.sinks import ExportSink from documents.export.sinks import ExportSink
from documents.export.sinks import StreamingManifestWriter from documents.export.sinks import StreamingManifestWriter
@@ -192,6 +197,28 @@ class Command(CryptMixin, PaperlessCommand):
help="Sets the export zip file name", help="Sets the export zip file name",
) )
parser.add_argument(
"--zip-compression",
choices=COMPRESSION_CHOICES,
default=None,
help=(
"Compression method for the export zip (requires --zip). "
"Default: deflated. 'zstd' requires Python 3.14+ on both the "
"exporting and importing machine."
),
)
parser.add_argument(
"--zip-compression-level",
type=int,
default=None,
help=(
"Compression level for the export zip (requires --zip). "
"deflated: 0-9, bzip2: 1-9, zstd: -22..22; ignored for "
"stored/lzma."
),
)
parser.add_argument( parser.add_argument(
"--data-only", "--data-only",
default=False, default=False,
@@ -247,12 +274,39 @@ class Command(CryptMixin, PaperlessCommand):
if not os.access(self.target, os.W_OK): if not os.access(self.target, os.W_OK):
raise CommandError("That path doesn't appear to be writable") raise CommandError("That path doesn't appear to be writable")
zip_compression: str | None = options["zip_compression"]
zip_compression_level: int | None = options["zip_compression_level"]
if not self.zip_export and (
zip_compression is not None or zip_compression_level is not None
):
raise CommandError(
"--zip-compression and --zip-compression-level require --zip",
)
compression_method = zip_compression or "deflated"
if self.zip_export:
if not compression_available(compression_method):
if compression_method == "zstd" and ZSTD is None:
raise CommandError(
"zstd compression requires Python 3.14 or newer",
)
raise CommandError(
f"Compression method '{compression_method}' is not "
f"available on this Python runtime",
)
level_msg = level_error(compression_method, zip_compression_level)
if level_msg is not None:
raise CommandError(level_msg)
sink: ExportSink sink: ExportSink
if self.zip_export: if self.zip_export:
sink = ZipExportSink( sink = ZipExportSink(
self.target, self.target,
options["zip_name"], options["zip_name"],
delete=self.delete, delete=self.delete,
compression=COMPRESSION_METHODS[compression_method],
compresslevel=zip_compression_level,
) )
else: else:
sink = DirectoryExportSink( sink = DirectoryExportSink(
@@ -32,6 +32,8 @@ from django.db.models.signals import post_save
from filelock import FileLock from filelock import FileLock
from guardian.shortcuts import clear_ct_cache from guardian.shortcuts import clear_ct_cache
from documents.export.compression import compress_type_readable
from documents.export.compression import unreadable_method_names
from documents.file_handling import create_source_path_directory from documents.file_handling import create_source_path_directory
from documents.management.commands.base import PaperlessCommand from documents.management.commands.base import PaperlessCommand
from documents.management.commands.mixins import CryptMixin from documents.management.commands.mixins import CryptMixin
@@ -460,6 +462,20 @@ class Command(CryptMixin, PaperlessCommand):
with tempfile.TemporaryDirectory() as tmp_dir: with tempfile.TemporaryDirectory() as tmp_dir:
if is_zipfile(self.source): if is_zipfile(self.source):
with ZipFile(self.source) as zf: with ZipFile(self.source) as zf:
unsupported = {
info.compress_type
for info in zf.infolist()
if not compress_type_readable(info.compress_type)
}
if unsupported:
names = sorted(unreadable_method_names(unsupported))
message = (
f"This archive uses compression this Python cannot "
f"read ({', '.join(names)})."
)
if "zstd" in names:
message += " zstd archives require Python 3.14+."
raise CommandError(message)
zf.extractall(tmp_dir) zf.extractall(tmp_dir)
self.source = Path(tmp_dir) self.source = Path(tmp_dir)
self._run_import() self._run_import()
+14 -10
View File
@@ -19,7 +19,7 @@ from documents.models import StoragePath
from documents.models import Tag from documents.models import Tag
from documents.models import Workflow from documents.models import Workflow
from documents.models import WorkflowTrigger from documents.models import WorkflowTrigger
from documents.permissions import permitted_object_ids from documents.permissions import get_objects_for_user_owner_aware
from documents.regex import safe_regex_search from documents.regex import safe_regex_search
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -55,8 +55,10 @@ def match_correspondents(document: Document, classifier: DocumentClassifier, use
user = document.owner user = document.owner
if user is not None: if user is not None:
correspondents = Correspondent.objects.filter( correspondents = get_objects_for_user_owner_aware(
id__in=permitted_object_ids(user, Correspondent, "view_correspondent"), user,
"documents.view_correspondent",
Correspondent,
) )
else: else:
correspondents = Correspondent.objects.all() correspondents = Correspondent.objects.all()
@@ -84,8 +86,10 @@ def match_document_types(document: Document, classifier: DocumentClassifier, use
user = document.owner user = document.owner
if user is not None: if user is not None:
document_types = DocumentType.objects.filter( document_types = get_objects_for_user_owner_aware(
id__in=permitted_object_ids(user, DocumentType, "view_documenttype"), user,
"documents.view_documenttype",
DocumentType,
) )
else: else:
document_types = DocumentType.objects.all() document_types = DocumentType.objects.all()
@@ -112,9 +116,7 @@ def match_tags(document: Document, classifier: DocumentClassifier, user=None):
user = document.owner user = document.owner
if user is not None: if user is not None:
tags = Tag.objects.filter( tags = get_objects_for_user_owner_aware(user, "documents.view_tag", Tag)
id__in=permitted_object_ids(user, Tag, "view_tag"),
)
else: else:
tags = Tag.objects.all() tags = Tag.objects.all()
@@ -143,8 +145,10 @@ def match_storage_paths(document: Document, classifier: DocumentClassifier, user
user = document.owner user = document.owner
if user is not None: if user is not None:
storage_paths = StoragePath.objects.filter( storage_paths = get_objects_for_user_owner_aware(
id__in=permitted_object_ids(user, StoragePath, "view_storagepath"), user,
"documents.view_storagepath",
StoragePath,
) )
else: else:
storage_paths = StoragePath.objects.all() storage_paths = StoragePath.objects.all()
+25 -59
View File
@@ -7,7 +7,6 @@ from django.contrib.contenttypes.models import ContentType
from django.db.models import Case from django.db.models import Case
from django.db.models import Count from django.db.models import Count
from django.db.models import IntegerField from django.db.models import IntegerField
from django.db.models import Model
from django.db.models import Q from django.db.models import Q
from django.db.models import QuerySet from django.db.models import QuerySet
from django.db.models import Value from django.db.models import Value
@@ -164,32 +163,30 @@ def set_permissions_for_object(
) )
def permitted_object_ids( def permitted_document_ids(
user: User | None, user,
model: type[Model],
perm: str,
*, *,
perm: str = "view_document",
include_deleted: bool = False, include_deleted: bool = False,
) -> QuerySet[int]: ):
""" """
Generic version of ``permitted_document_ids`` for any model with an Return a queryset of document IDs the user has ``perm`` on (default
``owner`` field and guardian object-level permissions. ``include_deleted`` ``"view_document"``). By default limited to non-deleted documents; pass
only has an effect for models exposing a ``global_objects``/``deleted_at`` ``include_deleted=True`` for callers that need to check permission on
soft-delete pattern (currently only ``Document``); for every other model soft-deleted documents (e.g. trash restore). This intentionally avoids
it is accepted but has no effect, since those models have no soft-delete ``get_objects_for_user`` to keep the subquery small and index-friendly.
concept.
""" """
has_soft_delete = hasattr(model, "global_objects")
manager = ( manager = Document.global_objects if include_deleted else Document.objects
model.global_objects if include_deleted and has_soft_delete else model.objects base_docs = manager.all()
) base_docs = base_docs.only("id", "owner")
base_qs = manager.all().only("id", "owner")
if user is None or not getattr(user, "is_authenticated", False): if user is None or not getattr(user, "is_authenticated", False):
return base_qs.filter(owner__isnull=True).values_list("id", flat=True) # Just Anonymous user e.g. for drf-spectacular
return base_docs.filter(owner__isnull=True).values_list("id", flat=True)
if getattr(user, "is_superuser", False): if getattr(user, "is_superuser", False):
return base_qs.values_list("id", flat=True) return base_docs.values_list("id", flat=True)
# Guardian's UserObjectPermission/GroupObjectPermission always store a bare # Guardian's UserObjectPermission/GroupObjectPermission always store a bare
# codename, but has_perm()-style callers commonly pass the qualified # codename, but has_perm()-style callers commonly pass the qualified
@@ -197,46 +194,31 @@ def permitted_object_ids(
# codename, so just drop any prefix rather than silently under-permitting. # codename, so just drop any prefix rather than silently under-permitting.
perm = perm.rsplit(".", 1)[-1] perm = perm.rsplit(".", 1)[-1]
content_type = ContentType.objects.get_for_model(model) document_ct = ContentType.objects.get_for_model(Document)
perm_filter = { perm_filter = {
"permission__codename": perm, "permission__codename": perm,
"permission__content_type": content_type, "permission__content_type": document_ct,
} }
user_perm_ids = ( user_perm_docs = (
UserObjectPermission.objects.filter(user=user, **perm_filter) UserObjectPermission.objects.filter(user=user, **perm_filter)
.annotate(object_pk_int=Cast("object_pk", IntegerField())) .annotate(object_pk_int=Cast("object_pk", IntegerField()))
.values_list("object_pk_int", flat=True) .values_list("object_pk_int", flat=True)
) )
group_perm_ids = (
group_perm_docs = (
GroupObjectPermission.objects.filter(group__user=user, **perm_filter) GroupObjectPermission.objects.filter(group__user=user, **perm_filter)
.annotate(object_pk_int=Cast("object_pk", IntegerField())) .annotate(object_pk_int=Cast("object_pk", IntegerField()))
.values_list("object_pk_int", flat=True) .values_list("object_pk_int", flat=True)
) )
permitted_ids = user_perm_ids.union(group_perm_ids)
return base_qs.filter( permitted_documents = user_perm_docs.union(group_perm_docs)
Q(owner=user) | Q(owner__isnull=True) | Q(id__in=permitted_ids),
return base_docs.filter(
Q(owner=user) | Q(owner__isnull=True) | Q(id__in=permitted_documents),
).values_list("id", flat=True) ).values_list("id", flat=True)
def permitted_document_ids(
user: User | None,
*,
perm: str = "view_document",
include_deleted: bool = False,
) -> QuerySet[int]:
"""
Document-specific convenience wrapper around ``permitted_object_ids``.
Return a queryset of document IDs the user has ``perm`` on (default
``"view_document"``). By default limited to non-deleted documents; pass
``include_deleted=True`` for callers that need to check permission on
soft-deleted documents (e.g. trash restore). This intentionally avoids
``get_objects_for_user`` to keep the subquery small and index-friendly.
"""
return permitted_object_ids(user, Document, perm, include_deleted=include_deleted)
def get_document_count_filter_for_user(user, related_name: str = "documents"): def get_document_count_filter_for_user(user, related_name: str = "documents"):
""" """
Return the Q object used to filter document counts for the given user. Return the Q object used to filter document counts for the given user.
@@ -359,13 +341,6 @@ def get_objects_for_user_owner_aware(
""" """
Returns objects the user owns, are unowned, or has explicit perms. Returns objects the user owns, are unowned, or has explicit perms.
When include_deleted is True, soft-deleted items are also included. When include_deleted is True, soft-deleted items are also included.
Legacy slow path (guardian-backed, O(n) style permission resolution).
Most queryset-filtering call sites have migrated onto
``PermittedObjectsFilter``/``permitted_object_ids()``, but this function
is kept because production callers still remain. Several callers remain
across ``documents/``, ``paperless_mail/``, and ``paperless_ai/`` --
grep for this function name before removing it.
""" """
manager = ( manager = (
Model.global_objects Model.global_objects
@@ -385,15 +360,6 @@ def get_objects_for_user_owner_aware(
def has_perms_owner_aware(user, perms, obj): def has_perms_owner_aware(user, perms, obj):
"""
Legacy slow path (guardian-backed) single-object permission check.
The queryset-filtering side of this migrated onto
``PermittedObjectsFilter``/``permitted_object_ids()``, but this
single-object check still has many production callers. Several callers
remain across ``documents/``, ``paperless_mail/``, and ``paperless_ai/``
-- grep for this function name before removing it.
"""
checker = ObjectPermissionChecker(user) checker = ObjectPermissionChecker(user)
return obj.owner is None or obj.owner == user or checker.has_perm(perms, obj) return obj.owner is None or obj.owner == user or checker.has_perm(perms, obj)
+3 -2
View File
@@ -70,7 +70,8 @@
] ]
</script> </script>
</pngx-root> </pngx-root>
<script src="{% static polyfills_js %}" type="module"></script> <script src="{% static runtime_js %}" defer></script>
<script src="{% static main_js %}" type="module"></script> <script src="{% static polyfills_js %}" defer></script>
<script src="{% static main_js %}" defer></script>
</body> </body>
</html> </html>
@@ -0,0 +1,190 @@
import sys
import zipfile
import pytest
from documents.export import compression
class TestCompressionMethods:
def test_choices_always_include_zstd(self) -> None:
"""
GIVEN:
- The compression policy module's CLI choices list
WHEN:
- Read on any runtime
THEN:
- zstd is always present; availability is checked separately so
argparse never hides it based on the current Python version
"""
assert compression.COMPRESSION_CHOICES == (
"stored",
"deflated",
"bzip2",
"lzma",
"zstd",
)
@pytest.mark.parametrize(
("name", "constant"),
[
("stored", zipfile.ZIP_STORED),
("deflated", zipfile.ZIP_DEFLATED),
("bzip2", zipfile.ZIP_BZIP2),
("lzma", zipfile.ZIP_LZMA),
],
)
def test_method_maps_to_zipfile_constant(self, name: str, constant: int) -> None:
"""
GIVEN:
- A compression method name
WHEN:
- Looked up in COMPRESSION_METHODS
THEN:
- It maps to the matching zipfile compression constant
"""
assert compression.COMPRESSION_METHODS[name] == constant
def test_stored_and_deflated_always_available(self) -> None:
"""
GIVEN:
- The stored and deflated compression methods
WHEN:
- Checked with compression_available()
THEN:
- Both are always available (zlib is a hard CPython dependency)
"""
assert compression.compression_available("stored")
assert compression.compression_available("deflated")
def test_zstd_availability_tracks_runtime(self) -> None:
"""
GIVEN:
- The zstd compression method
WHEN:
- Checked with compression_available() on this runtime
THEN:
- Availability matches whether Python is 3.14+
"""
expected: bool = sys.version_info >= (3, 14)
assert compression.compression_available("zstd") == expected
class TestLevelError:
@pytest.mark.parametrize(
("method", "level"),
[
("deflated", 0),
("deflated", 9),
("bzip2", 1),
("bzip2", 9),
("zstd", -22),
("zstd", 22),
("deflated", None),
("stored", None),
],
)
def test_valid_levels_return_none(self, method: str, level: int | None) -> None:
"""
GIVEN:
- A method and a level within its valid bounds (or no level)
WHEN:
- Checked with level_error()
THEN:
- No error message is returned
"""
assert compression.level_error(method, level) is None
@pytest.mark.parametrize(
("method", "level"),
[
("deflated", 10),
("deflated", -1),
("bzip2", 0),
("bzip2", 10),
("zstd", -23),
("zstd", 23),
],
)
def test_out_of_range_levels_return_message(
self,
method: str,
level: int,
) -> None:
"""
GIVEN:
- A method and a level outside its valid bounds
WHEN:
- Checked with level_error()
THEN:
- An error message naming the valid range is returned
"""
msg: str | None = compression.level_error(method, level)
assert msg is not None
assert "between" in msg
@pytest.mark.parametrize("method", ["stored", "lzma"])
def test_level_on_levelless_method_is_rejected(self, method: str) -> None:
"""
GIVEN:
- A method that ignores compression level (stored, lzma)
WHEN:
- A level is passed to level_error() anyway
THEN:
- An error message noting the level has no effect is returned
"""
msg: str | None = compression.level_error(method, 5)
assert msg is not None
assert "no effect" in msg
class TestCompressTypeReadable:
@pytest.mark.parametrize("ct", [zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED])
def test_stored_and_deflated_always_readable(self, ct: int) -> None:
"""
GIVEN:
- A stored or deflated compress_type id
WHEN:
- Checked with compress_type_readable()
THEN:
- It is always readable
"""
assert compression.compress_type_readable(ct)
def test_zstd_compress_type_readability_tracks_runtime(self) -> None:
"""
GIVEN:
- The current (93) and legacy (20) zstd compress_type ids
WHEN:
- Checked with compress_type_readable() on this runtime
THEN:
- Readability matches whether Python is 3.14+
"""
# 93 = ZIP_ZSTANDARD; 20 = legacy zstd method id (read-only)
expected: bool = sys.version_info >= (3, 14)
assert compression.compress_type_readable(93) == expected
assert compression.compress_type_readable(20) == expected
def test_unknown_compress_type_is_unreadable(self) -> None:
"""
GIVEN:
- An unrecognized compress_type id
WHEN:
- Checked with compress_type_readable()
THEN:
- It is reported as unreadable
"""
assert not compression.compress_type_readable(9999)
def test_unreadable_method_names_lists_methods(self) -> None:
"""
GIVEN:
- A set containing an unknown compress_type id
WHEN:
- Passed to unreadable_method_names()
THEN:
- It is reported generically as "method <id>"
"""
# An unknown method id maps to no name and is reported generically.
names: set[str] = compression.unreadable_method_names({9999})
assert names == {"method 9999"}
+43
View File
@@ -5,6 +5,7 @@ import zipfile
from pathlib import Path from pathlib import Path
import pytest import pytest
import pytest_mock
from pytest_django.fixtures import SettingsWrapper from pytest_django.fixtures import SettingsWrapper
from documents.export.sinks import DirectoryExportSink from documents.export.sinks import DirectoryExportSink
@@ -305,6 +306,48 @@ class TestZipExportSink:
assert not (target / "export.zip").exists() assert not (target / "export.zip").exists()
class TestZipExportSinkCompression:
@pytest.mark.parametrize(
("method", "constant"),
[
("stored", zipfile.ZIP_STORED),
("deflated", zipfile.ZIP_DEFLATED),
("bzip2", zipfile.ZIP_BZIP2),
("lzma", zipfile.ZIP_LZMA),
],
)
def test_compression_and_level_forwarded_to_zipfile(
self,
mocker: pytest_mock.MockerFixture,
tmp_path: Path,
method: str,
constant: int,
) -> None:
"""
GIVEN:
- A ZipExportSink constructed with a compression method and level
WHEN:
- The sink is opened
THEN:
- zipfile.ZipFile is constructed with those values forwarded
unchanged (whether ZipFile actually compresses is Python's own
contract, not ours, so this checks the call args, not a real
archive)
"""
target: Path = tmp_path / "out"
target.mkdir()
zip_cls = mocker.patch("documents.export.sinks.zipfile.ZipFile")
sink = ZipExportSink(target, "export", compression=constant, compresslevel=5)
sink._open()
zip_cls.assert_called_once_with(
mocker.ANY,
"w",
compression=constant,
compresslevel=5,
allowZip64=True,
)
class TestStreamContract: class TestStreamContract:
@pytest.fixture(params=["dir", "zip"]) @pytest.fixture(params=["dir", "zip"])
def sink(self, request: pytest.FixtureRequest, tmp_path: Path) -> ExportSink: def sink(self, request: pytest.FixtureRequest, tmp_path: Path) -> ExportSink:
+27 -46
View File
@@ -1057,52 +1057,33 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
THEN: THEN:
- The similar documents are returned from the API request - The similar documents are returned from the API request
""" """
# Distinct created/added/modified dates: documents sharing a timestamp # Distinct created/added dates: documents created at the same instant
# term (down to the second) would be matched on it by more_like_this # share a timestamp term, and more_like_this (which cannot be scoped to
# (which cannot be scoped to content fields), surfacing unrelated # content fields) would then match on it, surfacing unrelated documents.
# documents. `modified` is auto_now, so it can't be set via factory d1 = DocumentFactory(
# kwargs like created/added - freeze time per document instead so all title="invoice",
# three date fields land on distinct seconds. content="the thing i bought at a shop and paid with bank account",
with time_machine.travel( created=datetime.date(2018, 1, 1),
timezone.make_aware(datetime.datetime(2018, 1, 1)), added=timezone.make_aware(datetime.datetime(2018, 1, 1)),
tick=False, )
): d2 = DocumentFactory(
d1 = DocumentFactory( title="bank statement 1",
title="invoice", content="things i paid for in august",
content="the thing i bought at a shop and paid with bank account", created=datetime.date(2019, 3, 4),
created=datetime.date(2018, 1, 1), added=timezone.make_aware(datetime.datetime(2019, 3, 4)),
added=timezone.make_aware(datetime.datetime(2018, 1, 1)), )
) d3 = DocumentFactory(
with time_machine.travel( title="bank statement 3",
timezone.make_aware(datetime.datetime(2019, 3, 4)), content="things i paid for in september",
tick=False, created=datetime.date(2020, 7, 9),
): added=timezone.make_aware(datetime.datetime(2020, 7, 9)),
d2 = DocumentFactory( )
title="bank statement 1", d4 = DocumentFactory(
content="things i paid for in august", title="Quarterly Report",
created=datetime.date(2019, 3, 4), content="quarterly revenue profit margin earnings growth",
added=timezone.make_aware(datetime.datetime(2019, 3, 4)), created=datetime.date(2021, 11, 30),
) added=timezone.make_aware(datetime.datetime(2021, 11, 30)),
with time_machine.travel( )
timezone.make_aware(datetime.datetime(2020, 7, 9)),
tick=False,
):
d3 = DocumentFactory(
title="bank statement 3",
content="things i paid for in september",
created=datetime.date(2020, 7, 9),
added=timezone.make_aware(datetime.datetime(2020, 7, 9)),
)
with time_machine.travel(
timezone.make_aware(datetime.datetime(2021, 11, 30)),
tick=False,
):
d4 = DocumentFactory(
title="Quarterly Report",
content="quarterly revenue profit margin earnings growth",
created=datetime.date(2021, 11, 30),
added=timezone.make_aware(datetime.datetime(2021, 11, 30)),
)
backend = get_backend() backend = get_backend()
backend.add_or_update(d1) backend.add_or_update(d1)
backend.add_or_update(d2) backend.add_or_update(d2)
@@ -6,6 +6,8 @@ from datetime import timedelta
from io import StringIO from io import StringIO
from pathlib import Path from pathlib import Path
from unittest import mock from unittest import mock
from zipfile import ZIP_DEFLATED
from zipfile import ZIP_LZMA
from zipfile import ZipFile from zipfile import ZipFile
import pytest import pytest
@@ -1078,6 +1080,186 @@ class TestExportImport(
skip_checks=True, skip_checks=True,
) )
def test_compression_flags_require_zip(self) -> None:
"""
GIVEN:
- A request to export without --zip
WHEN:
- --zip-compression or --zip-compression-level is passed anyway
THEN:
- A CommandError is raised (the flags are meaningless without --zip)
"""
for args in (
["--zip-compression", "lzma"],
["--zip-compression-level", "5"],
):
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
*args,
skip_checks=True,
)
def test_zip_compression_level_out_of_range_raises(self) -> None:
"""
GIVEN:
- A request to export to a zip file
WHEN:
- --zip-compression-level is outside the chosen method's valid range
THEN:
- A CommandError is raised
"""
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"deflated",
"--zip-compression-level",
"99",
skip_checks=True,
)
def test_zip_compression_level_rejected_for_stored(self) -> None:
"""
GIVEN:
- A request to export to a zip file with --zip-compression stored
WHEN:
- --zip-compression-level is also passed
THEN:
- A CommandError is raised (stored ignores level entirely)
"""
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"stored",
"--zip-compression-level",
"5",
skip_checks=True,
)
def test_zip_compression_level_rejected_for_lzma(self) -> None:
"""
GIVEN:
- A request to export to a zip file with --zip-compression lzma
WHEN:
- --zip-compression-level is also passed
THEN:
- A CommandError is raised (lzma ignores level entirely)
"""
with self.assertRaises(CommandError):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"lzma",
"--zip-compression-level",
"5",
skip_checks=True,
)
def test_zstd_unavailable_raises_friendly_error(self) -> None:
"""
GIVEN:
- A Python runtime without zstd support (< 3.14)
WHEN:
- --zip-compression zstd is requested
THEN:
- A CommandError naming the Python version requirement is raised
zstd availability is mocked rather than relying on the actual
runtime: on a Python 3.14+ CI leg, ZSTD is not None, so without the
mock this check is skipped and the command falls through into the
real export, which fails on missing document files instead of
raising the expected CommandError.
"""
with (
mock.patch(
"documents.management.commands.document_exporter.ZSTD",
None,
),
mock.patch(
"documents.management.commands.document_exporter.compression_available",
return_value=False,
),
self.assertRaises(CommandError) as e,
):
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"zstd",
skip_checks=True,
)
self.assertIn("3.14", str(e.exception))
def test_zip_compression_flag_resolves_to_sink_constant(self) -> None:
"""
GIVEN:
- A request to export to a zip file with --zip-compression lzma
WHEN:
- The export runs
THEN:
- ZipExportSink is constructed with the resolved ZIP_LZMA constant
(whether zipfile actually compresses with the chosen method is
Python's own contract, and ZipExportSink's own tests already
cover the forwarding; what this command owns is resolving the
CLI string to the right constant, so assert that resolution
directly)
"""
with mock.patch(
"documents.management.commands.document_exporter.ZipExportSink",
) as sink_cls:
call_command(
"document_exporter",
self.target,
"--zip",
"--zip-compression",
"lzma",
skip_checks=True,
)
sink_cls.assert_called_once_with(
mock.ANY,
mock.ANY,
delete=False,
compression=ZIP_LZMA,
compresslevel=None,
)
def test_default_zip_compression_resolves_to_deflate(self) -> None:
"""
GIVEN:
- A request to export to a zip file with no --zip-compression flag
WHEN:
- The export runs
THEN:
- ZipExportSink is constructed with the default ZIP_DEFLATED
constant and compresslevel=None, matching pre-existing behavior
"""
with mock.patch(
"documents.management.commands.document_exporter.ZipExportSink",
) as sink_cls:
call_command(
"document_exporter",
self.target,
"--zip",
skip_checks=True,
)
sink_cls.assert_called_once_with(
mock.ANY,
mock.ANY,
delete=False,
compression=ZIP_DEFLATED,
compresslevel=None,
)
@pytest.mark.management @pytest.mark.management
class TestCryptExportImport( class TestCryptExportImport(
@@ -525,6 +525,35 @@ class TestCommandImport(
self.assertEqual(doc.tags.count(), 1) self.assertEqual(doc.tags.count(), 1)
self.assertEqual(doc.tags.first().name, "batch-flush-tag") self.assertEqual(doc.tags.first().name, "batch-flush-tag")
def test_import_rejects_unreadable_compression(self) -> None:
"""
GIVEN:
- A zip archive with an entry whose compression this Python can't read
WHEN:
- Import is attempted
THEN:
- A CommandError naming the issue is raised, before extraction
"""
import zipfile
from unittest import mock
archive = Path(self.dirs.scratch_dir) / "export.zip"
with zipfile.ZipFile(archive, "w") as zf:
zf.writestr("manifest.json", "[]")
with mock.patch(
"documents.management.commands.document_importer.compress_type_readable",
return_value=False,
):
with self.assertRaises(CommandError) as e:
call_command(
"document_importer",
str(archive),
"--no-progress-bar",
skip_checks=True,
)
self.assertIn("compression", str(e.exception))
@pytest.mark.management @pytest.mark.management
@pytest.mark.django_db @pytest.mark.django_db
@@ -12,22 +12,9 @@ from django.test import override_settings
from guardian.shortcuts import assign_perm from guardian.shortcuts import assign_perm
from rest_framework.test import APIClient from rest_framework.test import APIClient
from documents.matching import match_correspondents
from documents.matching import match_document_types
from documents.matching import match_storage_paths
from documents.matching import match_tags
from documents.models import Correspondent
from documents.models import DocumentType
from documents.models import StoragePath
from documents.models import Tag
from documents.permissions import permitted_document_ids from documents.permissions import permitted_document_ids
from documents.permissions import permitted_object_ids
from documents.serialisers import _get_viewable_duplicates from documents.serialisers import _get_viewable_duplicates
from documents.tests.factories import CorrespondentFactory
from documents.tests.factories import DocumentFactory from documents.tests.factories import DocumentFactory
from documents.tests.factories import DocumentTypeFactory
from documents.tests.factories import StoragePathFactory
from documents.tests.factories import TagFactory
def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden): def assert_visible_document_ids(actual_ids, *, expected_visible, expected_hidden):
@@ -444,320 +431,3 @@ class TestTrashRestorePermissionBoundary:
format="json", format="json",
) )
assert response.status_code == HTTPStatus.OK assert response.status_code == HTTPStatus.OK
@pytest.mark.django_db
class TestTrashViewExcludesExplicitlyGrantedDocuments:
"""
Regression test pinning TrashView's use of
``_TrashPermittedObjectsFilter`` (``include_granted = False``). If that
flag were ever flipped to the default ``True``, or the subclass removed
in favor of the base ``PermittedObjectsFilter``, a trashed document
would leak into ``/api/trash/`` results for any user holding an
explicit guardian grant on it, even though they are neither the owner
nor a superuser.
"""
def test_explicit_grant_does_not_leak_trashed_document(self, rest_api_client):
owner = User.objects.create_user(username="trash_owner")
grantee = User.objects.create_user(username="trash_grantee")
doc = DocumentFactory(owner=owner)
doc.delete() # soft delete
assign_perm("view_document", grantee, doc)
rest_api_client.force_authenticate(user=grantee)
response = rest_api_client.get("/api/trash/")
assert response.status_code == HTTPStatus.OK
result_ids = {result["id"] for result in response.data["results"]}
assert doc.pk not in result_ids
@pytest.mark.django_db
@pytest.mark.parametrize(
("model", "factory", "perm"),
[
(Tag, TagFactory, "view_tag"),
(Correspondent, CorrespondentFactory, "view_correspondent"),
(DocumentType, DocumentTypeFactory, "view_documenttype"),
(StoragePath, StoragePathFactory, "view_storagepath"),
],
)
class TestPermittedObjectIdsGenericModels:
def test_owner_sees_own_object(self, model, factory, perm):
owner = User.objects.create_user(username=f"owner_{model.__name__}")
stranger = User.objects.create_user(username=f"stranger_{model.__name__}")
owned = factory(owner=owner)
strangers = factory(owner=stranger)
assert_visible_document_ids(
permitted_object_ids(owner, model, perm),
expected_visible=[owned.pk],
expected_hidden=[strangers.pk],
)
def test_unowned_object_visible_to_everyone(self, model, factory, perm):
user = User.objects.create_user(username=f"user_{model.__name__}")
unowned = factory(owner=None)
assert_visible_document_ids(
permitted_object_ids(user, model, perm),
expected_visible=[unowned.pk],
expected_hidden=[],
)
def test_explicit_permission_grants_visibility(self, model, factory, perm):
owner = User.objects.create_user(username=f"owner2_{model.__name__}")
grantee = User.objects.create_user(username=f"grantee_{model.__name__}")
stranger = User.objects.create_user(username=f"stranger2_{model.__name__}")
shared = factory(owner=owner)
not_shared = factory(owner=owner)
assign_perm(perm, grantee, shared)
assert_visible_document_ids(
permitted_object_ids(grantee, model, perm),
expected_visible=[shared.pk],
expected_hidden=[not_shared.pk],
)
assert_visible_document_ids(
permitted_object_ids(stranger, model, perm),
expected_visible=[],
expected_hidden=[shared.pk, not_shared.pk],
)
def test_group_permission_grants_visibility_to_members_only(
self,
model,
factory,
perm,
):
owner = User.objects.create_user(username=f"owner3_{model.__name__}")
member = User.objects.create_user(username=f"member_{model.__name__}")
non_member = User.objects.create_user(username=f"nonmember_{model.__name__}")
group = Group.objects.create(name=f"group_{model.__name__}")
member.groups.add(group)
shared = factory(owner=owner)
assign_perm(perm, group, shared)
assert_visible_document_ids(
permitted_object_ids(member, model, perm),
expected_visible=[shared.pk],
expected_hidden=[],
)
assert_visible_document_ids(
permitted_object_ids(non_member, model, perm),
expected_visible=[],
expected_hidden=[shared.pk],
)
def test_superuser_sees_everything(self, model, factory, perm):
superuser = User.objects.create_superuser(username=f"root_{model.__name__}")
owner = User.objects.create_user(username=f"owner4_{model.__name__}")
obj = factory(owner=owner)
assert_visible_document_ids(
permitted_object_ids(superuser, model, perm),
expected_visible=[obj.pk],
expected_hidden=[],
)
@pytest.mark.django_db
class TestMatchingRespectsObjectPermissions:
def test_match_tags_only_considers_tags_visible_to_user(self):
owner = User.objects.create_user(username="tag_owner")
classifying_user = User.objects.create_user(username="classifier_user")
visible_tag = TagFactory(
owner=owner,
match="invoice",
matching_algorithm=Tag.MATCH_LITERAL,
)
hidden_tag = TagFactory(
owner=owner,
match="invoice",
matching_algorithm=Tag.MATCH_LITERAL,
)
assign_perm("view_tag", classifying_user, visible_tag)
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_tags(doc, classifier=None, user=classifying_user)
matched_ids = {t.pk for t in matched}
assert visible_tag.pk in matched_ids
assert hidden_tag.pk not in matched_ids
def test_match_correspondents_only_considers_correspondents_visible_to_user(self):
owner = User.objects.create_user(username="correspondent_owner")
classifying_user = User.objects.create_user(username="classifier_user2")
visible_correspondent = CorrespondentFactory(
owner=owner,
match="invoice",
matching_algorithm=Correspondent.MATCH_LITERAL,
)
hidden_correspondent = CorrespondentFactory(
owner=owner,
match="invoice",
matching_algorithm=Correspondent.MATCH_LITERAL,
)
assign_perm("view_correspondent", classifying_user, visible_correspondent)
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_correspondents(doc, classifier=None, user=classifying_user)
matched_ids = {c.pk for c in matched}
assert visible_correspondent.pk in matched_ids
assert hidden_correspondent.pk not in matched_ids
def test_match_document_types_only_considers_document_types_visible_to_user(self):
owner = User.objects.create_user(username="document_type_owner")
classifying_user = User.objects.create_user(username="classifier_user3")
visible_document_type = DocumentTypeFactory(
owner=owner,
match="invoice",
matching_algorithm=DocumentType.MATCH_LITERAL,
)
hidden_document_type = DocumentTypeFactory(
owner=owner,
match="invoice",
matching_algorithm=DocumentType.MATCH_LITERAL,
)
assign_perm("view_documenttype", classifying_user, visible_document_type)
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_document_types(doc, classifier=None, user=classifying_user)
matched_ids = {dt.pk for dt in matched}
assert visible_document_type.pk in matched_ids
assert hidden_document_type.pk not in matched_ids
def test_match_storage_paths_only_considers_storage_paths_visible_to_user(self):
owner = User.objects.create_user(username="storage_path_owner")
classifying_user = User.objects.create_user(username="classifier_user4")
visible_storage_path = StoragePathFactory(
owner=owner,
match="invoice",
matching_algorithm=StoragePath.MATCH_LITERAL,
)
hidden_storage_path = StoragePathFactory(
owner=owner,
match="invoice",
matching_algorithm=StoragePath.MATCH_LITERAL,
)
assign_perm("view_storagepath", classifying_user, visible_storage_path)
doc = DocumentFactory(owner=classifying_user, content="an invoice document")
matched = match_storage_paths(doc, classifier=None, user=classifying_user)
matched_ids = {sp.pk for sp in matched}
assert visible_storage_path.pk in matched_ids
assert hidden_storage_path.pk not in matched_ids
@pytest.mark.django_db
class TestBulkEditObjectsApplyToAllPermissionBoundary:
def test_apply_to_all_tags_excludes_unpermitted_tag(self, rest_api_client):
owner = User.objects.create_user(username="tags_owner")
requester = User.objects.create_user(username="tags_requester")
# grant the global change_tag permission so the object-level
# filtering (not the global has_perm check) is what's under test
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester)
visible = TagFactory(owner=owner)
hidden = TagFactory(owner=owner)
assign_perm("view_tag", requester, visible)
assign_perm("change_tag", requester, visible)
response = rest_api_client.post(
"/api/bulk_edit_objects/",
{
"object_type": "tags",
"operation": "set_permissions",
"all": True,
"filters": {},
"owner": requester.pk,
},
format="json",
)
assert response.status_code == HTTPStatus.OK
# The apply_to_all dispatch must resolve permitted objects up front:
# the visible tag (object-level change_tag granted) gets its owner
# reassigned, while the hidden tag (no object-level grant) is
# excluded entirely and keeps its original owner.
visible.refresh_from_db()
hidden.refresh_from_db()
assert visible.owner == requester
assert hidden.owner == owner
@pytest.mark.django_db
class TestBulkEditObjectsTagDescendantPartialPermission:
def test_apply_to_all_descendant_expansion_respects_per_object_permissions(
self,
rest_api_client,
):
"""
GIVEN:
- A tag hierarchy (parent -> permitted_child, unpermitted_child)
- A non-superuser requester with object-level change_tag granted
on the parent and on only ONE of the two children
WHEN:
- bulk_edit_objects is called with all=True and a filter that
matches only the root (parent) tag, engaging the
tag-descendant-expansion logic in BulkEditObjectsView.post
THEN:
- The descendant expansion only pulls in descendants the
requester actually has permission on: the permitted child's
owner is reassigned alongside the parent's, while the
unpermitted child keeps its original owner. This pins that the
expansion checks per-object permissions (editable_ids), not
merely "is a descendant of a filter match".
NOTE: this uses ``set_permissions`` (owner reassignment) rather than
``delete`` as the operation, because Tag.tn_parent (django-treenode)
cascades deletes to descendants at the database/ORM level regardless
of which tags the view resolved into ``objs`` -- a delete-based test
would pass/fail based on FK cascade behavior, not on whether the
descendant-expansion logic itself respected per-object permissions.
"""
owner = User.objects.create_user(username="tag_hierarchy_owner")
requester = User.objects.create_user(username="tag_hierarchy_requester")
# global change_tag permission so the has_perm() gate passes and the
# object-level permitted_object_ids filtering is what's under test
requester.user_permissions.add(
Permission.objects.get(codename="change_tag"),
)
rest_api_client.force_authenticate(user=requester)
parent = TagFactory(owner=owner, name="parent-tag")
permitted_child = TagFactory(
owner=owner,
name="permitted-child-tag",
tn_parent=parent,
)
unpermitted_child = TagFactory(
owner=owner,
name="unpermitted-child-tag",
tn_parent=parent,
)
assign_perm("change_tag", requester, parent)
assign_perm("change_tag", requester, permitted_child)
# unpermitted_child is intentionally NOT granted change_tag
response = rest_api_client.post(
"/api/bulk_edit_objects/",
{
"object_type": "tags",
"operation": "set_permissions",
"all": True,
"filters": {"is_root": True},
"owner": requester.pk,
},
format="json",
)
assert response.status_code == HTTPStatus.OK
parent.refresh_from_db()
permitted_child.refresh_from_db()
unpermitted_child.refresh_from_db()
assert parent.owner == requester
assert permitted_child.owner == requester
assert unpermitted_child.owner == owner
@@ -1,70 +0,0 @@
import pytest
from django.contrib.auth.models import User
from guardian.shortcuts import assign_perm
from rest_framework.test import APIRequestFactory
from documents.filters import PermittedObjectsFilter
from documents.models import Tag
from documents.tests.factories import TagFactory
class _DummyView:
queryset = Tag.objects.all()
@pytest.mark.django_db
class TestPermittedObjectsFilter:
def test_superuser_bypasses_filtering_entirely(self):
superuser = User.objects.create_superuser(username="root")
owner = User.objects.create_user(username="owner")
TagFactory(owner=owner)
request = APIRequestFactory().get("/")
request.user = superuser
result = PermittedObjectsFilter().filter_queryset(
request,
Tag.objects.all(),
_DummyView(),
)
assert result.count() == Tag.objects.count()
def test_non_superuser_sees_only_owned_unowned_and_granted(self):
owner = User.objects.create_user(username="owner")
grantee = User.objects.create_user(username="grantee")
owned = TagFactory(owner=grantee)
unowned = TagFactory(owner=None)
granted = TagFactory(owner=owner)
hidden = TagFactory(owner=owner)
assign_perm("view_tag", grantee, granted)
request = APIRequestFactory().get("/")
request.user = grantee
result = PermittedObjectsFilter().filter_queryset(
request,
Tag.objects.all(),
_DummyView(),
)
visible_ids = set(result.values_list("id", flat=True))
assert visible_ids == {owned.pk, unowned.pk, granted.pk}
assert hidden.pk not in visible_ids
def test_include_granted_false_excludes_explicitly_shared_objects(self):
owner = User.objects.create_user(username="owner2")
grantee = User.objects.create_user(username="grantee2")
owned = TagFactory(owner=grantee)
granted = TagFactory(owner=owner)
assign_perm("view_tag", grantee, granted)
request = APIRequestFactory().get("/")
request.user = grantee
class _OwnerOnlyFilter(PermittedObjectsFilter):
include_granted = False
result = _OwnerOnlyFilter().filter_queryset(
request,
Tag.objects.all(),
_DummyView(),
)
visible_ids = set(result.values_list("id", flat=True))
assert visible_ids == {owned.pk}
assert granted.pk not in visible_ids
+4
View File
@@ -78,6 +78,10 @@ class TestViews(DirectoriesMixin, TestCase):
response.context_data["styles_css"], response.context_data["styles_css"],
f"frontend/{language_actual}/styles.css", f"frontend/{language_actual}/styles.css",
) )
self.assertEqual(
response.context_data["runtime_js"],
f"frontend/{language_actual}/runtime.js",
)
self.assertEqual( self.assertEqual(
response.context_data["polyfills_js"], response.context_data["polyfills_js"],
f"frontend/{language_actual}/polyfills.js", f"frontend/{language_actual}/polyfills.js",
+19 -22
View File
@@ -133,10 +133,12 @@ from documents.file_handling import format_filename
from documents.filters import CorrespondentFilterSet from documents.filters import CorrespondentFilterSet
from documents.filters import CustomFieldFilterSet from documents.filters import CustomFieldFilterSet
from documents.filters import DocumentFilterSet from documents.filters import DocumentFilterSet
from documents.filters import DocumentPermissionsFilter
from documents.filters import DocumentsOrderingFilter from documents.filters import DocumentsOrderingFilter
from documents.filters import DocumentTypeFilterSet from documents.filters import DocumentTypeFilterSet
from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
from documents.filters import ObjectOwnedPermissionsFilter
from documents.filters import PaperlessTaskFilterSet from documents.filters import PaperlessTaskFilterSet
from documents.filters import PermittedObjectsFilter
from documents.filters import ShareLinkBundleFilterSet from documents.filters import ShareLinkBundleFilterSet
from documents.filters import ShareLinkFilterSet from documents.filters import ShareLinkFilterSet
from documents.filters import StoragePathFilterSet from documents.filters import StoragePathFilterSet
@@ -176,7 +178,6 @@ from documents.permissions import has_global_statistics_permission
from documents.permissions import has_perms_owner_aware from documents.permissions import has_perms_owner_aware
from documents.permissions import has_system_status_permission from documents.permissions import has_system_status_permission
from documents.permissions import permitted_document_ids from documents.permissions import permitted_document_ids
from documents.permissions import permitted_object_ids
from documents.permissions import set_permissions_for_object from documents.permissions import set_permissions_for_object
from documents.plugins.date_parsing import get_date_parser from documents.plugins.date_parsing import get_date_parser
from documents.schema import generate_object_with_permissions_schema from documents.schema import generate_object_with_permissions_schema
@@ -347,6 +348,7 @@ class IndexView(TemplateView):
context["username"] = self.request.user.username context["username"] = self.request.user.username
context["full_name"] = self.request.user.get_full_name() context["full_name"] = self.request.user.get_full_name()
context["styles_css"] = f"frontend/{self.get_frontend_language()}/styles.css" context["styles_css"] = f"frontend/{self.get_frontend_language()}/styles.css"
context["runtime_js"] = f"frontend/{self.get_frontend_language()}/runtime.js"
context["polyfills_js"] = ( context["polyfills_js"] = (
f"frontend/{self.get_frontend_language()}/polyfills.js" f"frontend/{self.get_frontend_language()}/polyfills.js"
) )
@@ -549,7 +551,7 @@ class CorrespondentViewSet(
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = CorrespondentFilterSet filterset_class = CorrespondentFilterSet
ordering_fields = ( ordering_fields = (
@@ -590,7 +592,7 @@ class TagViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Tag]):
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = TagFilterSet filterset_class = TagFilterSet
ordering_fields = ("color", "name", "matching_algorithm", "match", "document_count") ordering_fields = ("color", "name", "matching_algorithm", "match", "document_count")
@@ -682,7 +684,7 @@ class DocumentTypeViewSet(
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = DocumentTypeFilterSet filterset_class = DocumentTypeFilterSet
ordering_fields = ("name", "matching_algorithm", "match", "document_count") ordering_fields = ("name", "matching_algorithm", "match", "document_count")
@@ -986,7 +988,7 @@ class DocumentViewSet(
DjangoFilterBackend, DjangoFilterBackend,
SearchFilter, SearchFilter,
DocumentsOrderingFilter, DocumentsOrderingFilter,
PermittedObjectsFilter, DocumentPermissionsFilter,
) )
filterset_class = DocumentFilterSet filterset_class = DocumentFilterSet
search_fields = ("title", "correspondent__name", "effective_content") search_fields = ("title", "correspondent__name", "effective_content")
@@ -2672,7 +2674,7 @@ class SavedViewViewSet(BulkPermissionMixin, PassUserMixin, ModelViewSet[SavedVie
permission_classes = (IsAuthenticated, PaperlessObjectPermissions) permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
filter_backends = ( filter_backends = (
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
ordering_fields = ("name",) ordering_fields = ("name",)
@@ -3919,7 +3921,7 @@ class StoragePathViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Storag
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = StoragePathFilterSet filterset_class = StoragePathFilterSet
ordering_fields = ("name", "path", "matching_algorithm", "match", "document_count") ordering_fields = ("name", "path", "matching_algorithm", "match", "document_count")
@@ -4450,7 +4452,7 @@ class ShareLinkViewSet(
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = ShareLinkFilterSet filterset_class = ShareLinkFilterSet
ordering_fields = ("created", "expiration", "document") ordering_fields = ("created", "expiration", "document")
@@ -4480,7 +4482,7 @@ class ShareLinkBundleViewSet(PassUserMixin, ModelViewSet[ShareLinkBundle]):
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = ShareLinkBundleFilterSet filterset_class = ShareLinkBundleFilterSet
ordering_fields = ("created", "expiration", "status") ordering_fields = ("created", "expiration", "status")
@@ -4763,8 +4765,10 @@ class BulkEditObjectsView(PassUserMixin):
"document_types": DocumentTypeFilterSet, "document_types": DocumentTypeFilterSet,
"storage_paths": StoragePathFilterSet, "storage_paths": StoragePathFilterSet,
}[object_type] }[object_type]
user_permitted_objects = object_class.objects.filter( user_permitted_objects = get_objects_for_user_owner_aware(
id__in=permitted_object_ids(user, object_class, perm_codename), user,
perm_codename,
object_class,
) )
objs = filterset_class( objs = filterset_class(
data=filters, data=filters,
@@ -4789,11 +4793,8 @@ class BulkEditObjectsView(PassUserMixin):
if not user.is_superuser: if not user.is_superuser:
perm = f"documents.{perm_codename}" perm = f"documents.{perm_codename}"
has_perms = ( has_perms = user.has_perm(perm) and all(
user.has_perm(perm) has_perms_owner_aware(user, perm_codename, obj) for obj in objs
and not objs.exclude(
pk__in=permitted_object_ids(user, object_class, perm_codename),
).exists()
) )
if not has_perms: if not has_perms:
@@ -5294,11 +5295,7 @@ class SystemStatusView(PassUserMixin):
class TrashView(ListModelMixin, PassUserMixin): class TrashView(ListModelMixin, PassUserMixin):
permission_classes = (IsAuthenticated,) permission_classes = (IsAuthenticated,)
serializer_class = TrashSerializer serializer_class = TrashSerializer
filter_backends = (ObjectOwnedPermissionsFilter,)
class _TrashPermittedObjectsFilter(PermittedObjectsFilter):
include_granted = False
filter_backends = (_TrashPermittedObjectsFilter,)
pagination_class = StandardPagination pagination_class = StandardPagination
model = Document model = Document
+20 -20
View File
@@ -2,7 +2,7 @@ msgid ""
msgstr "" msgstr ""
"Project-Id-Version: paperless-ngx\n" "Project-Id-Version: paperless-ngx\n"
"Report-Msgid-Bugs-To: \n" "Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-08-08 14:28+0000\n" "POT-Creation-Date: 2026-08-05 14:50+0000\n"
"PO-Revision-Date: 2022-02-17 04:17\n" "PO-Revision-Date: 2022-02-17 04:17\n"
"Last-Translator: \n" "Last-Translator: \n"
"Language-Team: English\n" "Language-Team: English\n"
@@ -21,39 +21,39 @@ msgstr ""
msgid "Documents" msgid "Documents"
msgstr "" msgstr ""
#: documents/filters.py:471 #: documents/filters.py:472
msgid "Value must be valid JSON." msgid "Value must be valid JSON."
msgstr "" msgstr ""
#: documents/filters.py:490 #: documents/filters.py:491
msgid "Invalid custom field query expression" msgid "Invalid custom field query expression"
msgstr "" msgstr ""
#: documents/filters.py:500 #: documents/filters.py:501
msgid "Invalid expression list. Must be nonempty." msgid "Invalid expression list. Must be nonempty."
msgstr "" msgstr ""
#: documents/filters.py:521 #: documents/filters.py:522
msgid "Invalid logical operator {op!r}" msgid "Invalid logical operator {op!r}"
msgstr "" msgstr ""
#: documents/filters.py:535 #: documents/filters.py:536
msgid "Maximum number of query conditions exceeded." msgid "Maximum number of query conditions exceeded."
msgstr "" msgstr ""
#: documents/filters.py:599 #: documents/filters.py:600
msgid "{name!r} is not a valid custom field." msgid "{name!r} is not a valid custom field."
msgstr "" msgstr ""
#: documents/filters.py:636 #: documents/filters.py:637
msgid "{data_type} does not support query expr {expr!r}." msgid "{data_type} does not support query expr {expr!r}."
msgstr "" msgstr ""
#: documents/filters.py:755 documents/models.py:136 #: documents/filters.py:756 documents/models.py:136
msgid "Maximum nesting depth exceeded." msgid "Maximum nesting depth exceeded."
msgstr "" msgstr ""
#: documents/filters.py:1073 #: documents/filters.py:1098
msgid "Custom field not found" msgid "Custom field not found"
msgstr "" msgstr ""
@@ -1352,7 +1352,7 @@ msgid "workflow runs"
msgstr "" msgstr ""
#: documents/serialisers.py:521 documents/serialisers.py:873 #: documents/serialisers.py:521 documents/serialisers.py:873
#: documents/serialisers.py:2767 documents/views.py:299 documents/views.py:2555 #: documents/serialisers.py:2767 documents/views.py:300 documents/views.py:2557
#: paperless_mail/serialisers.py:155 #: paperless_mail/serialisers.py:155
msgid "Insufficient permissions." msgid "Insufficient permissions."
msgstr "" msgstr ""
@@ -1393,7 +1393,7 @@ msgstr ""
msgid "Duplicate document identifiers are not allowed." msgid "Duplicate document identifiers are not allowed."
msgstr "" msgstr ""
#: documents/serialisers.py:2853 documents/views.py:4509 #: documents/serialisers.py:2853 documents/views.py:4511
#, python-format #, python-format
msgid "Documents not found: %(ids)s" msgid "Documents not found: %(ids)s"
msgstr "" msgstr ""
@@ -1661,36 +1661,36 @@ msgstr ""
msgid "Unable to parse URI {value}" msgid "Unable to parse URI {value}"
msgstr "" msgstr ""
#: documents/views.py:292 documents/views.py:2552 #: documents/views.py:293 documents/views.py:2554
msgid "Invalid more_like_id" msgid "Invalid more_like_id"
msgstr "" msgstr ""
#: documents/views.py:1566 #: documents/views.py:1568
msgid "Invalid AI configuration." msgid "Invalid AI configuration."
msgstr "" msgstr ""
#: documents/views.py:1575 #: documents/views.py:1577
msgid "AI backend request timed out." msgid "AI backend request timed out."
msgstr "" msgstr ""
#: documents/views.py:2377 documents/views.py:2698 #: documents/views.py:2379 documents/views.py:2700
msgid "Specify only one of text, title_search, query, or more_like_id." msgid "Specify only one of text, title_search, query, or more_like_id."
msgstr "" msgstr ""
#: documents/views.py:4522 #: documents/views.py:4524
#, python-format #, python-format
msgid "Insufficient permissions to share document %(id)s." msgid "Insufficient permissions to share document %(id)s."
msgstr "" msgstr ""
#: documents/views.py:4568 #: documents/views.py:4570
msgid "Bundle is already being processed." msgid "Bundle is already being processed."
msgstr "" msgstr ""
#: documents/views.py:4629 #: documents/views.py:4631
msgid "The share link bundle is still being prepared. Please try again later." msgid "The share link bundle is still being prepared. Please try again later."
msgstr "" msgstr ""
#: documents/views.py:4639 #: documents/views.py:4641
msgid "The share link bundle is unavailable." msgid "The share link bundle is unavailable."
msgstr "" msgstr ""
+5 -11
View File
@@ -21,7 +21,6 @@ from typing import Self
from django.conf import settings from django.conf import settings
from documents.parsers import ParseError
from paperless.version import __full_version_str__ from paperless.version import __full_version_str__
if TYPE_CHECKING: if TYPE_CHECKING:
@@ -367,7 +366,8 @@ class RemoteDocumentParser:
"""Send ``file`` to Azure AI Document Intelligence and return text. """Send ``file`` to Azure AI Document Intelligence and return text.
Downloads the searchable PDF output from Azure and stores it at Downloads the searchable PDF output from Azure and stores it at
``self._archive_path``. ``self._archive_path``. Returns the extracted text content, or
``None`` on failure (the error is logged).
Parameters Parameters
---------- ----------
@@ -379,14 +379,7 @@ class RemoteDocumentParser:
Returns Returns
------- -------
str | None str | None
Extracted text. Extracted text, or None if the Azure call failed.
Raises
------
ParseError
If the Azure call fails for any reason. The error is logged
and re-raised so consumption fails loudly instead of silently
producing a document with no content.
""" """
if TYPE_CHECKING: if TYPE_CHECKING:
# Callers must have already validated config via engine_is_valid(): # Callers must have already validated config via engine_is_valid():
@@ -433,7 +426,8 @@ class RemoteDocumentParser:
except Exception as e: except Exception as e:
logger.exception("Azure AI Vision parsing failed: %s", e) logger.exception("Azure AI Vision parsing failed: %s", e)
raise ParseError(f"Azure AI Vision parsing failed: {e}") from e
finally: finally:
client.close() client.close()
return None
-1
View File
@@ -217,7 +217,6 @@ class ApplicationConfigurationSerializer(
llm_api_key = ObfuscatedPasswordField( llm_api_key = ObfuscatedPasswordField(
required=False, required=False,
allow_null=True, allow_null=True,
max_length=1024,
) )
def run_validation(self, data): def run_validation(self, data):
@@ -20,7 +20,6 @@ from unittest.mock import Mock
import pytest import pytest
from documents.parsers import ParseError
from paperless.parsers import ParserContext from paperless.parsers import ParserContext
from paperless.parsers import ParserProtocol from paperless.parsers import ParserProtocol
from paperless.parsers.remote import RemoteDocumentParser from paperless.parsers.remote import RemoteDocumentParser
@@ -343,14 +342,15 @@ class TestRemoteParserParse:
class TestRemoteParserParseError: class TestRemoteParserParseError:
def test_parse_raises_parse_error_on_azure_error( def test_parse_returns_empty_on_azure_error(
self, self,
remote_parser: RemoteDocumentParser, remote_parser: RemoteDocumentParser,
simple_digital_pdf_file: Path, simple_digital_pdf_file: Path,
failing_azure_client: Mock, failing_azure_client: Mock,
) -> None: ) -> None:
with pytest.raises(ParseError, match="Azure AI Vision parsing failed"): remote_parser.parse(simple_digital_pdf_file, "application/pdf")
remote_parser.parse(simple_digital_pdf_file, "application/pdf")
assert remote_parser.get_text() == ""
def test_parse_closes_client_on_error( def test_parse_closes_client_on_error(
self, self,
@@ -358,8 +358,7 @@ class TestRemoteParserParseError:
simple_digital_pdf_file: Path, simple_digital_pdf_file: Path,
failing_azure_client: Mock, failing_azure_client: Mock,
) -> None: ) -> None:
with pytest.raises(ParseError): remote_parser.parse(simple_digital_pdf_file, "application/pdf")
remote_parser.parse(simple_digital_pdf_file, "application/pdf")
failing_azure_client.close.assert_called_once() failing_azure_client.close.assert_called_once()
@@ -372,8 +371,7 @@ class TestRemoteParserParseError:
) -> None: ) -> None:
mock_log = mocker.patch("paperless.parsers.remote.logger") mock_log = mocker.patch("paperless.parsers.remote.logger")
with pytest.raises(ParseError): remote_parser.parse(simple_digital_pdf_file, "application/pdf")
remote_parser.parse(simple_digital_pdf_file, "application/pdf")
mock_log.exception.assert_called_once() mock_log.exception.assert_called_once()
assert "Azure AI Vision parsing failed" in mock_log.exception.call_args[0][0] assert "Azure AI Vision parsing failed" in mock_log.exception.call_args[0][0]
-27
View File
@@ -757,30 +757,3 @@ class TestAPIProcessedMails(DirectoriesMixin, APITestCase):
format="json", format="json",
) )
self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST) self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
def test_bulk_delete_processed_mails_rejects_mixed_batch_atomically(self) -> None:
"""
GIVEN:
- A permitted processed mail and one the user may not delete
WHEN:
- API call bulk deletes both in a single request
THEN:
- The request is rejected and neither mail is deleted
"""
user2 = User.objects.create_user(username="temp_admin2")
rule = MailRuleFactory()
# Created first so it sorts ahead of the forbidden mail, i.e. the
# permission check has to cover the whole batch before deleting rather
# than rejecting only once it reaches the forbidden one.
pm_owned = ProcessedMailFactory(rule=rule, owner=self.user)
pm_forbidden = ProcessedMailFactory(rule=rule, owner=user2)
response = self.client.post(
f"{self.ENDPOINT}bulk_delete/",
data={"mail_ids": [pm_owned.id, pm_forbidden.id]},
format="json",
)
self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
self.assertTrue(ProcessedMail.objects.filter(id=pm_owned.id).exists())
self.assertTrue(ProcessedMail.objects.filter(id=pm_forbidden.id).exists())
+8 -16
View File
@@ -23,11 +23,10 @@ from rest_framework.response import Response
from rest_framework.viewsets import ModelViewSet from rest_framework.viewsets import ModelViewSet
from rest_framework.viewsets import ReadOnlyModelViewSet from rest_framework.viewsets import ReadOnlyModelViewSet
from documents.filters import PermittedObjectsFilter from documents.filters import ObjectOwnedOrGrantedPermissionsFilter
from documents.models import PaperlessTask from documents.models import PaperlessTask
from documents.permissions import PaperlessObjectPermissions from documents.permissions import PaperlessObjectPermissions
from documents.permissions import has_perms_owner_aware from documents.permissions import has_perms_owner_aware
from documents.permissions import permitted_object_ids
from documents.views import PassUserMixin from documents.views import PassUserMixin
from paperless.views import StandardPagination from paperless.views import StandardPagination
from paperless_mail.filters import ProcessedMailFilterSet from paperless_mail.filters import ProcessedMailFilterSet
@@ -76,7 +75,7 @@ class MailAccountViewSet(PassUserMixin, ModelViewSet[MailAccount]):
serializer_class = MailAccountSerializer serializer_class = MailAccountSerializer
pagination_class = StandardPagination pagination_class = StandardPagination
permission_classes = (IsAuthenticated, PaperlessObjectPermissions) permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
filter_backends = (PermittedObjectsFilter,) filter_backends = (ObjectOwnedOrGrantedPermissionsFilter,)
def get_permissions(self): def get_permissions(self):
if self.action == "test": if self.action == "test":
@@ -198,7 +197,7 @@ class ProcessedMailViewSet(PassUserMixin, ReadOnlyModelViewSet[ProcessedMail]):
filter_backends = ( filter_backends = (
DjangoFilterBackend, DjangoFilterBackend,
OrderingFilter, OrderingFilter,
PermittedObjectsFilter, ObjectOwnedOrGrantedPermissionsFilter,
) )
filterset_class = ProcessedMailFilterSet filterset_class = ProcessedMailFilterSet
@@ -212,17 +211,10 @@ class ProcessedMailViewSet(PassUserMixin, ReadOnlyModelViewSet[ProcessedMail]):
): ):
return HttpResponseBadRequest("mail_ids must be a list of integers") return HttpResponseBadRequest("mail_ids must be a list of integers")
mails = ProcessedMail.objects.filter(id__in=mail_ids) mails = ProcessedMail.objects.filter(id__in=mail_ids)
# Check every id up front so an unpermitted one rejects the whole for mail in mails:
# request rather than deleting the mails ahead of it first. if not has_perms_owner_aware(request.user, "delete_processedmail", mail):
if mails.exclude( return HttpResponseForbidden("Insufficient permissions")
pk__in=permitted_object_ids( mail.delete()
request.user,
ProcessedMail,
"delete_processedmail",
),
).exists():
return HttpResponseForbidden("Insufficient permissions")
mails.delete()
return Response({"result": "OK", "deleted_mail_ids": mail_ids}) return Response({"result": "OK", "deleted_mail_ids": mail_ids})
@@ -233,7 +225,7 @@ class MailRuleViewSet(PassUserMixin, ModelViewSet[MailRule]):
serializer_class = MailRuleSerializer serializer_class = MailRuleSerializer
pagination_class = StandardPagination pagination_class = StandardPagination
permission_classes = (IsAuthenticated, PaperlessObjectPermissions) permission_classes = (IsAuthenticated, PaperlessObjectPermissions)
filter_backends = (PermittedObjectsFilter,) filter_backends = (ObjectOwnedOrGrantedPermissionsFilter,)
@extend_schema_view( @extend_schema_view(
Generated
+592 -576
View File
File diff suppressed because it is too large Load Diff