Commit Graph

3955 Commits

Author SHA1 Message Date
stumpylog
39534d2227 test(tasks): fix two spec gaps in task API test suite
Move test_list_is_owner_aware to TestGetTasksV10 (it tests GET /api/tasks/,
not acknowledge). Add test_related_document_ids_includes_duplicate_of to
cover the duplicate_of path in the related_document_ids property.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
bc63ca5806 test(tasks): rewrite API task tests for redesigned model and v9 compat
Replaces the old Django TestCase-based tests with pytest-style classes using
PaperlessTaskFactory. Covers v10 field names, v9 backwards-compat field
mapping, filtering, ordering, acknowledge, acknowledge_all, summary, active,
and run endpoints. Also adds PaperlessTaskFactory to factories.py and fixes
a redundant source= kwarg in TaskSerializerV10.related_document_ids.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
1ae146a38b fix(tasks): add read_only_fields to TaskSerializerV9, enforce admin via permission_classes on run action 2026-04-15 18:34:34 -07:00
stumpylog
39c7a04dd2 feat(tasks): update serializer, filter, and viewset with v9 backwards compat
- Replace TasksViewSerializer/RunTaskViewSerializer with TaskSerializerV10
  (new field names), TaskSerializerV9 (v9 compat), TaskSummarySerializer,
  and RunTaskSerializer
- Add AcknowledgeTasksViewSerializer unchanged (kept existing validation)
- Expand PaperlessTaskFilterSet with MultipleChoiceFilter for task_type,
  trigger_source, status; add is_complete, date_created_after/before filters
- Replace TasksViewSet.get_serializer_class() to branch on request.version
- Add get_queryset() v9 compat for task_name/type query params
- Add acknowledge_all, summary, active actions to TasksViewSet
- Rewrite run action to use apply_async with trigger_source header
- Add timedelta import to views.py; add MultipleChoiceFilter/DateTimeFilter
  to filters.py imports

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
868ddbc2fa feat(tasks): auto-inject trigger_source=scheduled header for all beat tasks
Inject `headers: {"trigger_source": "scheduled"}` into every Celery beat
schedule entry so signal handlers can identify scheduler-originated tasks
without per-task instrumentation.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
b241250741 refactor(tasks): remove manual PaperlessTask creation and scheduled/auto params
All task records are now created exclusively via Celery signals (Task 2).
Removed PaperlessTask creation/update from train_classifier, sanity_check,
llmindex_index, and check_sanity. Removed scheduled= and auto= parameters
from all 7 call sites. Updated apply_async callers to use trigger_source
headers instead. Exceptions now propagate naturally from task functions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
efeb5b447c test(tasks): add traceback and revoked state coverage to signal tests 2026-04-15 18:34:34 -07:00
stumpylog
ad47e96df1 feat(tasks): rewrite signal handlers to track all task types
Replace the old consume_file-only handler with a full rewrite that tracks
6 task types (consume_file, train_classifier, sanity_check, index_optimize,
llm_index, mail_fetch) with proper trigger source detection, input data
extraction, legacy result string parsing, duration/wait time recording,
and structured error capture on failure.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
stumpylog
ef4e3d31ef feat(tasks): replace PaperlessTask model with structured redesign
Drop the old string-based PaperlessTask table and recreate it with
Status/TaskType/TriggerSource enums, JSONField result storage, and
duration tracking fields. Update all call sites to use the new API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 18:34:34 -07:00
GitHub Actions
dc06b679d3 Auto translate strings 2026-04-15 23:22:00 +00:00
Trenton H
3ffbb8862c Feature: paginate search highlights and remove 10k document search limit (#12518)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-04-15 23:20:31 +00:00
Trenton H
21db608d57 Feature: Tune SQLite default settings for increased speed (#12580) 2026-04-15 15:26:00 -07:00
Trenton H
2fd1a1cf3a Feature: Document fuzzy match improvements (#12579)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 20:59:43 +00:00
GitHub Actions
2b4c1fe20d Auto translate strings 2026-04-15 18:58:56 +00:00
Gaëtan GOUZI
02008e40d8 fix: Return HTTP 400 instead of HTTP 500 on DELETE /api/documents/{id}/notes/ with missing or invalid note id (#12582) 2026-04-15 11:56:15 -07:00
GitHub Actions
d9061d5e55 Auto translate strings 2026-04-14 22:15:38 +00:00
shamoon
8f036c9521 Merge branch 'main' into dev
# Conflicts:
#	docs/usage.md
#	src/documents/signals/handlers.py
#	src/documents/tests/test_api_documents.py
#	src/documents/views.py
2026-04-14 15:11:23 -07:00
Trenton H
17c13c1a03 Breaking: Remove the positional arguments from the pre/post consume scripts. Environment has been preferred for quite some time now (#12573) 2026-04-14 21:18:20 +00:00
shamoon
ffd886eae0 Bump version to 2.20.14 2026-04-14 13:11:48 -07:00
shamoon
12c0dc635e Merge branch 'release/v2.20.x' 2026-04-14 13:10:52 -07:00
GitHub Actions
3ed7297939 Auto translate strings 2026-04-13 21:14:40 +00:00
Trenton H
3b6edcdd8e Chore: Add generic type params and update our baselines (#12566)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 14:12:59 -07:00
GitHub Actions
b27d10646e Auto translate strings 2026-04-13 20:12:04 +00:00
Trenton H
8c1225e120 Fixes an N+1 query in matching with the version content fetching by prefetching versions (#12562) 2026-04-13 13:10:28 -07:00
Trenton H
54d5269145 Fix: Use an iterator in the sanity checking (#12563) 2026-04-13 12:32:22 -07:00
shamoon
df3b656352 Add tests 2026-04-10 12:06:28 -07:00
shamoon
51e721733f Enhancement: validate and sanitize uploaded logos (#12551) 2026-04-10 11:50:58 -07:00
GitHub Actions
ec6969e326 Auto translate strings 2026-04-08 15:42:05 +00:00
shamoon
4629bbf83e Enhancement: add view_global_statistics and view_system_status permissions (#12530) 2026-04-08 15:39:47 +00:00
GitHub Actions
51c59746a7 Auto translate strings 2026-04-06 22:51:57 +00:00
Trenton H
c232d443fa Breaking: Decouple OCR control from archive file control (#12448)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-04-06 15:50:21 -07:00
Trenton H
5f5fb263c9 Fix: Don't create a new note highlight generator per note in the loop (#12512) 2026-04-03 17:34:15 -07:00
shamoon
b807b107ad Enhancement: include sharelinks + bundles in export/import (#12479) 2026-04-03 21:51:57 +00:00
Trenton H
c2f02851da Chore: Better typed status manager messages (#12509) 2026-04-03 21:18:01 +00:00
GitHub Actions
d0f8a98a9a Auto translate strings 2026-04-03 20:55:14 +00:00
shamoon
566afdffca Enhancement: unify text search to use tantivy (#12485) 2026-04-03 13:53:45 -07:00
Trenton H
f32ad98d8e Feature: Update consumer logging to include task ID for log correlation (#12510) 2026-04-03 13:31:40 -07:00
Trenton H
d365f19962 Security: Registers a custom serializer which signs the task payload (#12504) 2026-04-03 03:49:54 +00:00
GitHub Actions
2703c12f1a Auto translate strings 2026-04-03 03:25:57 +00:00
shamoon
e7c7978d67 Enhancement: allow opt-in blocking internal mail hosts (#12502) 2026-04-03 03:24:28 +00:00
GitHub Actions
83501757df Auto translate strings 2026-04-02 22:36:32 +00:00
Trenton H
dda05a7c00 Security: Improve overall security in a few ways (#12501)
- Make sure we're always using regex with timeouts for user controlled data
- Adds rate limiting to the token endpoint (configurable)
- Signs the classifier pickle file with the SECRET_KEY and refuse to load one which doesn't verify.
- Require the user to set a secret key, instead of falling back to our old hard coded one
2026-04-02 15:30:26 -07:00
Trenton H
376af81b9c Fix: Resolve another TC assuming an object has been created somewhere (#12503) 2026-04-02 14:58:28 -07:00
GitHub Actions
05c9e21fac Auto translate strings 2026-04-02 19:40:05 +00:00
Trenton H
aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
2026-04-02 12:38:22 -07:00
shamoon
2f5bcdf66e Fix: dont defer tag change application in workflows (#12478) 2026-04-02 11:54:37 -07:00
GitHub Actions
2aa0c9f0b4 Auto translate strings 2026-03-31 18:25:03 +00:00
shamoon
d2328b776a Performance: support bulk edit without id lists (#12355) 2026-03-31 18:23:28 +00:00
GitHub Actions
e1da2a1efe Auto translate strings 2026-03-31 14:57:34 +00:00
shamoon
245514ad10 Performance: deprecate and remove usage of all in API results (#12309) 2026-03-31 07:55:59 -07:00