Commit Graph

3942 Commits

Author SHA1 Message Date
Trenton H
8e4ad33702 feat(tasks): update TaskAdmin for redesigned model
Add date_created, duration_seconds to list_display; add trigger_source
to list_filter; add input_data, duration_seconds, wait_time_seconds to
readonly_fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 10:36:19 -07:00
Trenton H
48db462c2c test(tasks): address code quality review findings
Remove trivial field-existence tests per project conventions. Fix
potentially flaky ordering test to use explicit date_created values.
Add is_complete=false filter test, v9 type filter input direction test,
and tighten TestActive second test to target REVOKED specifically.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 10:34:32 -07:00
Trenton H
e4f11d6cfa test(tasks): fix two spec gaps in task API test suite
Move test_list_is_owner_aware to TestGetTasksV10 (it tests GET /api/tasks/,
not acknowledge). Add test_related_document_ids_includes_duplicate_of to
cover the duplicate_of path in the related_document_ids property.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 10:19:11 -07:00
Trenton H
bedb965b84 test(tasks): rewrite API task tests for redesigned model and v9 compat
Replaces the old Django TestCase-based tests with pytest-style classes using
PaperlessTaskFactory. Covers v10 field names, v9 backwards-compat field
mapping, filtering, ordering, acknowledge, acknowledge_all, summary, active,
and run endpoints. Also adds PaperlessTaskFactory to factories.py and fixes
a redundant source= kwarg in TaskSerializerV10.related_document_ids.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 10:07:38 -07:00
Trenton H
901237962b fix(tasks): add read_only_fields to TaskSerializerV9, enforce admin via permission_classes on run action 2026-04-15 07:25:58 -07:00
Trenton H
c4794c0fe7 feat(tasks): update serializer, filter, and viewset with v9 backwards compat
- Replace TasksViewSerializer/RunTaskViewSerializer with TaskSerializerV10
  (new field names), TaskSerializerV9 (v9 compat), TaskSummarySerializer,
  and RunTaskSerializer
- Add AcknowledgeTasksViewSerializer unchanged (kept existing validation)
- Expand PaperlessTaskFilterSet with MultipleChoiceFilter for task_type,
  trigger_source, status; add is_complete, date_created_after/before filters
- Replace TasksViewSet.get_serializer_class() to branch on request.version
- Add get_queryset() v9 compat for task_name/type query params
- Add acknowledge_all, summary, active actions to TasksViewSet
- Rewrite run action to use apply_async with trigger_source header
- Add timedelta import to views.py; add MultipleChoiceFilter/DateTimeFilter
  to filters.py imports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 06:58:42 -07:00
Trenton H
425c32122a feat(tasks): auto-inject trigger_source=scheduled header for all beat tasks
Inject `headers: {"trigger_source": "scheduled"}` into every Celery beat
schedule entry so signal handlers can identify scheduler-originated tasks
without per-task instrumentation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 20:56:06 -07:00
Trenton H
f5c0834ce3 refactor(tasks): remove manual PaperlessTask creation and scheduled/auto params
All task records are now created exclusively via Celery signals (Task 2).
Removed PaperlessTask creation/update from train_classifier, sanity_check,
llmindex_index, and check_sanity. Removed scheduled= and auto= parameters
from all 7 call sites. Updated apply_async callers to use trigger_source
headers instead. Exceptions now propagate naturally from task functions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 16:55:17 -07:00
Trenton H
bc561135e5 test(tasks): add traceback and revoked state coverage to signal tests 2026-04-14 16:19:23 -07:00
Trenton H
c306a4ffe7 feat(tasks): rewrite signal handlers to track all task types
Replace the old consume_file-only handler with a full rewrite that tracks
6 task types (consume_file, train_classifier, sanity_check, index_optimize,
llm_index, mail_fetch) with proper trigger source detection, input data
extraction, legacy result string parsing, duration/wait time recording,
and structured error capture on failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 15:41:45 -07:00
Trenton H
0780b7e506 feat(tasks): replace PaperlessTask model with structured redesign
Drop the old string-based PaperlessTask table and recreate it with
Status/TaskType/TriggerSource enums, JSONField result storage, and
duration tracking fields. Update all call sites to use the new API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 15:29:38 -07:00
Trenton H
17c13c1a03 Breaking: Remove the positional arguments from the pre/post consume scripts. Environment has been preferred for quite some time now (#12573) 2026-04-14 21:18:20 +00:00
GitHub Actions
3ed7297939 Auto translate strings 2026-04-13 21:14:40 +00:00
Trenton H
3b6edcdd8e Chore: Add generic type params and update our baselines (#12566)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 14:12:59 -07:00
GitHub Actions
b27d10646e Auto translate strings 2026-04-13 20:12:04 +00:00
Trenton H
8c1225e120 Fixes an N+1 query in matching with the version content fetching by prefetching versions (#12562) 2026-04-13 13:10:28 -07:00
Trenton H
54d5269145 Fix: Use an iterator in the sanity checking (#12563) 2026-04-13 12:32:22 -07:00
shamoon
df3b656352 Add tests 2026-04-10 12:06:28 -07:00
shamoon
51e721733f Enhancement: validate and sanitize uploaded logos (#12551) 2026-04-10 11:50:58 -07:00
GitHub Actions
ec6969e326 Auto translate strings 2026-04-08 15:42:05 +00:00
shamoon
4629bbf83e Enhancement: add view_global_statistics and view_system_status permissions (#12530) 2026-04-08 15:39:47 +00:00
GitHub Actions
51c59746a7 Auto translate strings 2026-04-06 22:51:57 +00:00
Trenton H
c232d443fa Breaking: Decouple OCR control from archive file control (#12448)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-04-06 15:50:21 -07:00
Trenton H
5f5fb263c9 Fix: Don't create a new note highlight generator per note in the loop (#12512) 2026-04-03 17:34:15 -07:00
shamoon
b807b107ad Enhancement: include sharelinks + bundles in export/import (#12479) 2026-04-03 21:51:57 +00:00
Trenton H
c2f02851da Chore: Better typed status manager messages (#12509) 2026-04-03 21:18:01 +00:00
GitHub Actions
d0f8a98a9a Auto translate strings 2026-04-03 20:55:14 +00:00
shamoon
566afdffca Enhancement: unify text search to use tantivy (#12485) 2026-04-03 13:53:45 -07:00
Trenton H
f32ad98d8e Feature: Update consumer logging to include task ID for log correlation (#12510) 2026-04-03 13:31:40 -07:00
Trenton H
d365f19962 Security: Registers a custom serializer which signs the task payload (#12504) 2026-04-03 03:49:54 +00:00
GitHub Actions
2703c12f1a Auto translate strings 2026-04-03 03:25:57 +00:00
shamoon
e7c7978d67 Enhancement: allow opt-in blocking internal mail hosts (#12502) 2026-04-03 03:24:28 +00:00
GitHub Actions
83501757df Auto translate strings 2026-04-02 22:36:32 +00:00
Trenton H
dda05a7c00 Security: Improve overall security in a few ways (#12501)
- Make sure we're always using regex with timeouts for user controlled data
- Adds rate limiting to the token endpoint (configurable)
- Signs the classifier pickle file with the SECRET_KEY and refuse to load one which doesn't verify.
- Require the user to set a secret key, instead of falling back to our old hard coded one
2026-04-02 15:30:26 -07:00
Trenton H
376af81b9c Fix: Resolve another TC assuming an object has been created somewhere (#12503) 2026-04-02 14:58:28 -07:00
GitHub Actions
05c9e21fac Auto translate strings 2026-04-02 19:40:05 +00:00
Trenton H
aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
2026-04-02 12:38:22 -07:00
GitHub Actions
2aa0c9f0b4 Auto translate strings 2026-03-31 18:25:03 +00:00
shamoon
d2328b776a Performance: support bulk edit without id lists (#12355) 2026-03-31 18:23:28 +00:00
GitHub Actions
e1da2a1efe Auto translate strings 2026-03-31 14:57:34 +00:00
shamoon
245514ad10 Performance: deprecate and remove usage of all in API results (#12309) 2026-03-31 07:55:59 -07:00
GitHub Actions
020057e1a4 Auto translate strings 2026-03-30 16:40:47 +00:00
shamoon
f715533770 Performance: support passing selection data with filtered document requests (#12300) 2026-03-30 16:38:52 +00:00
Jan Kleine
0292edbee7 Fixhancement: include trashed documents in document exporter/importer (#12425) 2026-03-30 16:30:22 +00:00
Andreas Schneider
85e0d1842a Tests: add regression test for redis URL with empty username (#12460)
* Tests: add regression test for redis URL with empty username and password

Covers the unix://:SECRET@/path.sock format (empty username, password only),
which was missing from the existing test cases for PR #12239.

* Update src/paperless/tests/settings/test_custom_parsers.py

---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-03-29 06:31:18 -07:00
GitHub Actions
62f79c088e Auto translate strings 2026-03-28 21:00:05 +00:00
shamoon
129da3ade7 Tweakhancement: show file extension in StoragePath test (#12452) 2026-03-28 13:58:33 -07:00
Trenton H
9383471fa0 Feature: Transition all checksums to use SHA256 (#12432) 2026-03-26 11:28:02 -07:00
GitHub Actions
b153ec803b Auto translate strings 2026-03-26 14:38:10 +00:00
shamoon
ae0474450f Chore: logger, response and template sanitization cleanup (#12439) 2026-03-26 07:36:02 -07:00