amorfo77/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2026-09-13 13:17:59 +00:00
Code Issues Projects Releases Wiki Activity
12,101 Commits 32 Branches 231 Tags
55209dedb56ceb4f308838ec0de5cbdb3f978fd5
Commit Graph
70 Commits
Author SHA1 Message Date
Trenton HandClaude Opus 5 60709b8319 Performance: cut redundant per-document lookups in bulk modify_custom_fields (#13807) 
* Perf: batch the repeated lookups in modify_custom_fields

`modify_custom_fields()` re-resolved the same objects inside its
per-document loop: `custom_fields.get(id=field_id)` re-ran a CustomField
query for every document, and doc link fields called
`Document.objects.get(id=doc_id)` a second time for a document that was
already known.

Resolve both up front with `in_bulk()` and hand the resolved objects to
`update_or_create()` rather than bare ids. Passing the objects also
populates the FK cache on the newly created instance, so auditlog's
post_save receiver touching `.document`/`.field` no longer costs a reload
per row. The document map defers `content`, the one field here that is
both large and unused. The symmetrical-link removal pass and
`remove_doclink()` get `select_related()` for the same auditlog reason.

Measured over 50 documents, sqlite, audit log enabled:

                        before   after
  add 3 string fields    1502    1054
  update 1 string field   451     403
  add doc link            851     653
  remove doc link         604     354

`update_or_create()` is kept as-is. Dropping it for a hand-rolled
get-or-construct loop removes a further ~4 statements per row, but those
are the SAVEPOINT/RELEASE pairs of its `transaction.atomic()`, and the
`select_for_update()` and IntegrityError fallback that go with them. The
(document, field) unique constraint depends on that when two bulk edits
overlap, and the wall clock did not move to pay for it (331 ms vs 310 ms
for the string case above).

Also normalises the field ids to int once at the top so the old dict API,
whose keys may arrive as strings, indexes the resolved map correctly.

The `if custom_field:` branch it replaces was dead: `.get()` raises
DoesNotExist, it never returns None.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* Fixes the comment

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
 2026-09-09 23:47:07 +00:00
shamoon 8d1bc5dd24 Fix: prevent orphaned versions from bulk delete (#14030) 2026-09-09 08:48:59 -07:00
shamoon c93c996edf Remote ocr reprocess (#13636) 
* Backend stuff for remote ocr reprocess, add to bulk edit pass in from ui settings

* Ok, frontend reprocess remote option

* Docs
 2026-08-24 14:43:05 -07:00
shamoon e389298aab Enhancement: merge documents as versions (#13515) 2026-08-19 07:20:14 -07:00
shamoon 6f8e39c2e0 Fix: avoid unnecessary creating new PDF with pw removal workflow (#12948) 2026-06-07 20:30:08 +00:00
shamoon 0a6e0db186 Fix: use chord.on_error before apply_async (#12842) 2026-05-24 14:42:11 -07:00
Trenton Handshamoon bd86dca57e Fix: Password removal source file location (#12830) 
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
 2026-05-19 13:52:04 -07:00
Trenton H c3b353873b use a few named tuples and data classes instead of so much unpacking (#12709) 2026-05-04 18:00:48 -07:00
Trenton H 5010f37174 Fix: avoid unnecessary close_old_connections in Celery task dispatch (#12701) 2026-05-04 06:02:28 -07:00
Trenton H fbf4e32646 Chore: Converts all call sites and test asserts to use apply_async and headers (#12591) 2026-04-20 11:40:04 -07:00
Trenton HClaude Sonnet 4.6Antoine Mérino
 		aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471) 
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
 2026-04-02 12:38:22 -07:00
shamoon 866c9fd858 Fix: correct merge bulk edit indentation 2026-03-15 23:50:54 -07:00
shamoon 731448a8f9 Fixhancement: support version-specific edits (#12233) 2026-03-06 11:59:26 -08:00
shamoon ceee769e26 Feature: document file versions (#12061) 2026-02-26 16:46:54 +00:00
shamoon 6192915be7 Fixhancement: improve ASN handling with PDF operations (#11689) 2026-02-06 21:14:02 +00:00
Sebastian Steinbeißer 3b5ffbf9fa Chore(mypy): Annotate None returns for typing improvements (#11213) 2026-02-02 08:44:12 -08:00
shamoon 5b1e66be91 Feature: password removal action (#11656) 2026-01-08 21:36:11 +00:00
shamoon 7604a0b583 Fix: prevent ASN collisions for merge operations (#11634) 2025-12-19 20:05:34 -08:00
shamoonandTrenton H 4cff907ba0 Feature: Nested Tags (#10833) 
---------

Co-authored-by: Trenton H <797416+stumpylog@users.noreply.github.com>
 2025-09-17 21:41:39 +00:00
shamoon 44b8c4881a Fix: fix error when bulk adding empty doc link custom fields (#10832) 2025-09-11 13:19:23 -07:00
shamoon 6d72ee795f Feature: PDF editor (#10318) 2025-08-11 17:29:48 +00:00
shamoon 6dea158de9 Fix: prevent self-linking when bulk edit doc link (#9629) 2025-04-14 07:12:50 -07:00
shamoon 1e489a0666 Enhancement: add switch to allow merging non-PDFs with archive version (#9305) 2025-03-05 20:46:51 +00:00
Trenton H 827fcba277 Chore: Reduce imports for a slight memory improvement (#9217) 2025-02-24 15:06:14 -08:00
Sebastian Steinbeißer e560fa3be0 Chore: Enable ruff FBT (#8645) 2025-02-07 09:12:03 -08:00
shamoon 52ab07c673 Fix: deselect and trigger refresh for deleted documents from bulk operations with "delete originals" (#8996) 2025-02-07 00:47:50 +00:00
shamoon 54e72d5b60 Fix: also ensure symmetric doc link removal on bulk edit (#8963) 2025-01-31 07:39:22 -08:00
shamoon 5b8c9ef5fc Fix: reflect doc links in bulk modify custom fields (#8962) 2025-01-30 10:55:05 -08:00
shamoon 765bf1d11c Fix: use hard delete when bulk editing custom fields (#8740) 2025-01-14 16:47:29 -08:00
shamoon e4f69dc945 Feature: bulk edit custom field values (#8428) 2024-12-09 17:35:49 +00:00
shamoon 9c1561adfb Change: change update content to handle archive disabled (#8315) 2024-11-20 20:01:13 +00:00
Sebastian Steinbeißer 74d0c9fda5 Switch src/documents/bulk*.py from os.path to pathlib.Path (#7862) 
Also:
* Ensure that the ruff PTH check remains enabled for these files and
all files added in the future.
* Add some type annotations.
 2024-11-12 17:04:07 +00:00
shamoon 28fdb170bf Fix: handle uuid fields created under mariadb and Django 4 (#8034) 2024-10-28 13:54:16 +00:00
Trenton H e6f59472e4 Chore: Drop Python 3.9 support (#7774) 2024-09-26 12:22:24 -07:00
s0llvanandshamoon c92c3e224a Feature: page count (#7750) 
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
 2024-09-25 08:22:12 -07:00
shamoon a64d457c30 Enhancement: use request user as owner of split / merge docs (#7112) 2024-06-27 13:46:49 -07:00
shamoon f3cf608caa Fix: use temp dir for split / merge (#7105) 2024-06-26 10:46:59 -07:00
Dominik Bruhnandshamoon d1ac15baa9 Enhancement: support delete originals after split / merge (#6935) 
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
 2024-06-08 11:29:03 -07:00
shamoon d558367642 Change: rename redo OCR to reprocess (#6866) 2024-05-31 18:41:52 +00:00
shamoon 3e62f13f96 Enhancement: delete pages PDF action (#6772) 2024-05-22 23:01:15 +00:00
Trenton H 622f624132 Chore: Change the code formatter to Ruff (#6756) 
* Changing the formatting to ruff-format

* Replaces references to black to ruff or ruff format, removes black from dependencies
 2024-05-18 02:26:50 +00:00
shamoon caec0ed4d1 Fix: allow bulk edit with existing fields (#6625) 2024-05-07 14:26:07 -07:00
Trenton H 7be7185418 Handcrafts SQL queries a little more to reduce the query count and/or the amount of returned data (#6489) 2024-04-30 07:37:09 -07:00
shamoon 63e1f9f5d3 Feature: custom fields filtering & bulk editing (#6484) 2024-04-26 15:10:03 -07:00
Trenton H ae2b302962 Fix: Rotation fails due to celery chord (#6306) 
Use Celery immutable signature so it does not take the result of the header task
 2024-04-06 23:26:48 +00:00
shamoon 4af8070450 Feature: PDF actions - merge, split & rotate (#6094) 2024-03-25 18:41:24 -07:00
shamoon 454098630b Enhancement: mergeable bulk edit permissions (#5508) 2024-02-01 19:56:57 +00:00
Trenton H 6f163111ce Upgrades black to v23, upgrades ruff 2023-04-26 09:35:27 -07:00
Trenton H 3bcbd05252 Fixes ruff not running isort against the codebase 2023-04-26 09:35:27 -07:00
Jonas Winkler c87ca25f22 fix database locked error 2023-03-03 23:30:12 +01:00