Commit Graph
73 Commits
Author SHA1 Message Date
JaydenandGitHub 879cd4a30a Enhancement: Add --url argument to document_fuzzy_match to improve output (#13123)
Added a new --url argument to specify the base URL of the Paperless instance, allowing matched documents to be displayed as clickable links. Updated the logic to fetch document titles based on the presence of the base URL.
2026-08-12 08:23:05 -07:00
Trenton HandGitHub 486b4babac Performance: sqlite-vec point-delete for document chunks (#13438)
* Refactor: extract migration infrastructure, add has_pending_migration() (#13410)

* Empty commit to try and get Codecov going
2026-07-31 19:10:59 -07:00
Trenton HandGitHub 1646756c6c Docs: warn bare-metal users about stale files when upgrading (#13296) 2026-07-25 14:06:31 -07:00
bb5d7438b1 Documentation (beta): Updates documentation for new v3 features (#13033)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-06-18 16:20:31 -07:00
e822e72964 Feature: Further reduce document importer memory usage (#12707)
* Replaces loaddata with streaming bulk_create

Replaces call_command('loaddata') with a streaming implementation that
reads manifest records one at a time via ijson, accumulates per-model
batches up to --batch-size, and flushes via bulk_create.  This reduces
peak memory and no longer scales directly with the size of the import.

* fix(importer): avoid guardian lru_cache poisoning; include M2M through tables in check_constraints

clear_cache() inside the import transaction emptied Django's ContentType
manager cache while fixture PKs were live, causing downstream ContentType
lookups to repopulate guardian's separate @lru_cache(None) with
fixture-PK objects. After the TestCase transaction rolled back to
original PKs, guardian's lru_cache held stale fixture ContentType
objects, causing MixedContentTypeError in unrelated subsequent tests.

Remove clear_cache() since it was defending against a theoretical
stale-cache scenario that doesn't occur in a proper same-install restore.

Fix check_constraints() to explicitly include auto-created M2M through
tables (populated by .set() after bulk_create) alongside the model tables,
addressing the gap where join-table FK violations would have gone
undetected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* Excludes the consumer and AnonymousUser from any models which might have a FK relation to it.  This prevents orphan things like UI setting, which have a relation to no existing user

* Splits into more sub functions for Sonar

* Improvements to the typing of the new functions

* Coverage for some error cases, and removes handling for pk only models.  No need to support these

* Final coverage gaps

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-04 16:36:05 -07:00
aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
2026-04-02 12:38:22 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>shamoon
e2947ccff2 Chore(deps): Bump the pre-commit-dependencies group with 4 updates (#12323)
* Chore(deps): Bump the pre-commit-dependencies group with 4 updates

---
updated-dependencies:
- dependency-name: https://github.com/codespell-project/codespell
  dependency-version: 2.4.2
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
- dependency-name: prettier
  dependency-version: 3.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pre-commit-dependencies
- dependency-name: prettier-plugin-organize-imports
  dependency-version: 4.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: pre-commit-dependencies
- dependency-name: https://github.com/lovesegfault/beautysh
  dependency-version: 6.4.3
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Drop this, it seems more trouble than its worth

* Re-run prek with new prettier

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-03-12 16:29:57 +00:00
Trenton HandGitHub 5988d5896b Breaking: Refactor advanced database settings to allow more user configuration (#12165) 2026-02-27 14:37:26 -08:00
shamoon 3281ec2401 Documentation: update duplicates note 2026-01-28 08:25:16 -08:00
Trenton HandGitHub d0032c18be Breaking: Remove support for document and thumbnail encryption (#11850) 2026-01-24 19:29:54 -08:00
shamoonandGitHub bf468ac64f Documentation: fix cache invalidation command in documentation (#10665) 2025-08-23 04:00:02 -07:00
shamoonandGitHub f00a565130 Documentation: fix Postgres version 2025-08-17 01:33:52 -07:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>shamoon
edb8c06e2a Chore(deps): Bump the django group across 1 directory with 9 updates (#10538)
* Chore(deps): Bump the django group across 1 directory with 9 updates

Bumps the django group with 9 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [django](https://github.com/django/django) | `5.1.8` | `5.2.5` |
| [django-auditlog](https://github.com/jazzband/django-auditlog) | `3.1.2` | `3.2.1` |
| [django-guardian](https://github.com/django-guardian/django-guardian) | `2.4.0` | `3.0.3` |
| [django-multiselectfield](https://github.com/goinnn/django-multiselectfield) | `0.1.13` | `1.0.1` |
| [django-soft-delete](https://github.com/san4ezy/django_softdelete) | `1.0.18` | `1.0.19` |
| [djangorestframework](https://github.com/encode/django-rest-framework) | `3.16.0` | `3.16.1` |
| [djangorestframework-guardian](https://github.com/rpkilby/django-rest-framework-guardian) | `0.3.0` | `0.4.0` |
| [drf-spectacular-sidecar](https://github.com/tfranzel/drf-spectacular-sidecar) | `2025.4.1` | `2025.8.1` |
| [pytest-django](https://github.com/pytest-dev/pytest-django) | `4.10.0` | `4.11.1` |



Updates `django` from 5.1.8 to 5.2.5
- [Commits](https://github.com/django/django/compare/5.1.8...5.2.5)

Updates `django-auditlog` from 3.1.2 to 3.2.1
- [Release notes](https://github.com/jazzband/django-auditlog/releases)
- [Changelog](https://github.com/jazzband/django-auditlog/blob/master/CHANGELOG.md)
- [Commits](https://github.com/jazzband/django-auditlog/compare/v3.1.2...v3.2.1)

Updates `django-guardian` from 2.4.0 to 3.0.3
- [Release notes](https://github.com/django-guardian/django-guardian/releases)
- [Commits](https://github.com/django-guardian/django-guardian/compare/v2.4.0...3.0.3)

Updates `django-multiselectfield` from 0.1.13 to 1.0.1
- [Release notes](https://github.com/goinnn/django-multiselectfield/releases)
- [Changelog](https://github.com/goinnn/django-multiselectfield/blob/master/CHANGES.rst)
- [Commits](https://github.com/goinnn/django-multiselectfield/compare/v0.1.13...v1.0.1)

Updates `django-soft-delete` from 1.0.18 to 1.0.19
- [Changelog](https://github.com/san4ezy/django_softdelete/blob/master/CHANGELOG.md)
- [Commits](https://github.com/san4ezy/django_softdelete/commits)

Updates `djangorestframework` from 3.16.0 to 3.16.1
- [Release notes](https://github.com/encode/django-rest-framework/releases)
- [Commits](https://github.com/encode/django-rest-framework/compare/3.16.0...3.16.1)

Updates `djangorestframework-guardian` from 0.3.0 to 0.4.0
- [Changelog](https://github.com/rpkilby/django-rest-framework-guardian/blob/master/CHANGELOG)
- [Commits](https://github.com/rpkilby/django-rest-framework-guardian/compare/0.3.0...0.4.0)

Updates `drf-spectacular-sidecar` from 2025.4.1 to 2025.8.1
- [Commits](https://github.com/tfranzel/drf-spectacular-sidecar/compare/2025.4.1...2025.8.1)

Updates `pytest-django` from 4.10.0 to 4.11.1
- [Release notes](https://github.com/pytest-dev/pytest-django/releases)
- [Changelog](https://github.com/pytest-dev/pytest-django/blob/main/docs/changelog.rst)
- [Commits](https://github.com/pytest-dev/pytest-django/compare/v4.10.0...v4.11.1)

---
updated-dependencies:
- dependency-name: django
  dependency-version: 5.2.5
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django
- dependency-name: django-auditlog
  dependency-version: 3.2.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django
- dependency-name: django-guardian
  dependency-version: 3.0.3
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: django
- dependency-name: django-multiselectfield
  dependency-version: 1.0.1
  dependency-type: direct:production
  update-type: version-update:semver-major
  dependency-group: django
- dependency-name: django-soft-delete
  dependency-version: 1.0.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: django
- dependency-name: djangorestframework
  dependency-version: 3.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: django
- dependency-name: djangorestframework-guardian
  dependency-version: 0.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django
- dependency-name: drf-spectacular-sidecar
  dependency-version: 2025.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django
- dependency-name: pytest-django
  dependency-version: 4.11.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: django
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix log matches related to newlines, add newlines to stdout.writelines

* Fix disable api remote auth test, Django 5.2 no longer uses process_request

* Remove postgres version check

* Update administration.md

* Handle django-multiselectfield v1.0 changes

* Update administration.md

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-08-11 13:45:14 -07:00
Katrin LeinweberandGitHub 5410074062 Documentation: copy-edits (#10417) 2025-07-20 17:27:04 +00:00
1671d49d44 Enhancement: Add a database caching for improved performance (#9784)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-06-30 22:36:24 -07:00
KilianandGitHub 246f17c6c8 Enhancement: support import of zipped export (#10073) 2025-06-13 10:06:37 -07:00
shamoonandGitHub 32a7f9cd5a Enhancement: allow webUI first account signup (#9500) 2025-03-29 17:12:34 +00:00
shamoon 6a8ec182fa Documentation: clarify encryption docs 2025-02-08 08:07:04 -08:00
Malte BreitzmannandGitHub 2a1c6bf0ca Documentation: remove dollar sign in code snippets (#8600) 2025-01-04 00:30:25 +00:00
shamoonandGitHub 7d182ab894 Enhancement: prune audit logs and management command (#8416) 2024-12-03 19:28:27 +00:00
Kevin DorenandGitHub 827121808a Enhancement: Add --compare-json option to document_exporter to write json files only if changed (#8261) 2024-11-19 07:20:24 -08:00
shamoonandGitHub aac04e73b9 Fix: correct serializing of auth tokens for export (#8100) 2024-10-29 17:02:32 +00:00
ad8c60d153 Documentation: add no-progress-bar option to importer/exporter (#7477)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-08-17 02:57:03 +00:00
d9002005b1 Feature: Allow encrypting sensitive fields in export (#6927)
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-06-09 14:41:18 +00:00
Trenton HandGitHub 5fc6736666 Fixes up various doc sections (#6883) 2024-06-02 03:18:12 +00:00
Trenton HandGitHub 085447e7c4 Feature: Allow a data only export/import cycle (#6871) 2024-06-01 18:22:59 -07:00
a712bc72ca Documentation: document_importer does not directly support zip archives (#6209)
---------

Co-authored-by: Sander <biocoder-xbmc@xs4all.nl>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2024-03-27 19:53:04 -07:00
Sebastian SteinbeißerandGitHub f6c34494a7 Documentation: change updating instructions (#5533) 2024-01-25 13:21:36 -08:00
Trenton HandGitHub 7289c4ea56 Feature: Allow deletion of documents via the fuzzy matching command (#4957)
* Adds new flag allowing deletion of one of a document pair which is over the match ratio

* Documents the new command option
2023-12-17 18:37:38 -08:00
omahsandGitHub ce643942ea Documentation: Fix typos (#4737) 2023-12-01 14:55:03 +00:00
c8ee35692c Documentation: Update documentation to refer only to Docker Compose v2 command (#4650)
* Replaces references to docker-compose (the v1 executable) with docker compose (the v2 plugin) as well as fixing up some referenes between the tool vs the command

* Update docs/setup.md

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>

* Replaces references to docker-compose (the v1 executable) with docker compose (the v2 plugin) as well as fixing up some referenes between the tool vs the command

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-11-20 20:14:33 -08:00
0d92648d62 Documentation: Structure backup section more clearly (#4559)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2023-11-16 00:26:52 +00:00
shamoon 3239c478a5 Merge branch 'main' into dev 2023-11-13 19:57:08 -08:00
shamoonandGitHub 55f26c9c4c Documentation: update docs, screenshots ahead of Paperless-ngx v2.0 (#4542)
* Reformat internal links

* Update screenshots, docs & readme

* Use image lightbox

* Fix nested list on docs index

* Add some visual interest to screenshots layout

* Compress images
2023-11-13 14:56:34 +00:00
Trenton HandGitHub e8527ba723 Chore: Cleanup command arguments and standardize process count handling (#4541)
Cleans up some command help text and adds more control over process count for command with a Pool
2023-11-09 11:46:37 -08:00
RockandGitHub 5b83bd03f5 Documentation: clarify document_exporter includes settings (#4533) 2023-11-09 06:20:27 -08:00
shamoon d9abb745a9 Merge branch 'main' into dev 2023-09-26 13:33:27 -07:00
shamoonandGitHub 379b4f8cd3 Documentation enhancement: add direct links for all config vars (#4237) 2023-09-22 11:12:47 -07:00
Trenton HandGitHub ec9ebd3026 Allow the user the specifiy the zip file name (#4189) 2023-09-15 16:33:28 -07:00
Trenton H 81b9f2d4e0 Cleans up the docs, adds validation of the process count, include the test descriptions 2023-09-12 08:17:12 -07:00
Trenton H a03a745295 Adds multiprocessing to the fuzzy matching for a speedup 2023-09-12 08:17:12 -07:00
Trenton Holmes e2ae919a84 Implements a new command for fuzzy matching document content and reporting potential duplicates 2023-09-12 08:17:12 -07:00
b238ba054d Enhancement: add --id-range for document_retagger (#4080)
---------

Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
2023-09-08 17:33:24 +00:00
mechanarchyandshamoon 7ef81ae10f Updates documentation formatting, grammar, links
Correct indentation of lists, code blocks

Update faq.md

Update CONTRIBUTING.md

Update development.md

Update advanced_usage.md

Update administration.md

Update configuration.md

Update configuration.md

Update advanced_usage.md

Update faq.md
2023-07-21 10:17:57 -07:00
Trenton H c4407dccf6 Updates the default Postgres to 15 for new installs 2023-06-20 10:35:48 -07:00
andstu a05dbd2e5a fix: spelling 2023-05-25 00:43:31 -04:00
Trenton H 3cdd358fc8 Adds a note to bare metal upgrading about removing old dependencies 2023-05-16 12:05:29 -07:00
Trenton H d246e4090a Adds a tip for using tokens instead of passwords, now that support is integrated 2023-03-23 08:52:12 -07:00
Max Truxa 8161316c01 Fix: Update outdated docs referencing master
The `master` branch is now called `main`.
2023-03-14 10:19:00 +01:00
Trenton Holmes 7ecb76dddc Merge remote-tracking branch 'origin/main' into beta 2023-01-24 17:54:42 -08:00