mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-07-28 06:44:57 +00:00
fa01396dfd
* Fix: selection_data re-derives the filtered document set 5 times over _get_selection_data_for_queryset() (powers ?include_selection_data=true on the document list and search endpoints) computed document_count for Correspondent/DocumentType/StoragePath/Tag/CustomField by embedding the caller's full filtered queryset -- filters plus the permission check -- as a subquery inside 5 separate Count(filter=Q(documents__in=queryset), ...) calls. Each one re-evaluates that whole queryset from scratch. Resolve the document ids once into a concrete list and reuse it across all five annotations instead. For Tag/CustomField specifically (M2M via a through-model table), also route through annotate_document_count_by_ids() -- extracted from the tag/custom-field document_count fix (#13203) -- to avoid the same Count(filter=Q(id__in=...), distinct=True)-on-an-M2M-relation anti-pattern diagnosed there. On a 400k-document/1,000-tag corpus, the correspondents portion alone previously didn't finish within several minutes (killed twice while investigating, including one run that left a zombie query still consuming a CPU 30+ minutes later). All five queries together now complete in ~30-35s. Root-caused from a real report (paperless-ngx#13201) via a different, already-fixed query (#13205) -- this one hasn't been reported in the wild yet, found by auditing the same call path. Depends on #13203 for annotate_document_count_by_ids(). Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * Address review feedback: drop unnecessary ordering before collecting ids queryset passed into _get_selection_data_for_queryset() carries the default/user-specified ordering, which is irrelevant once we're only collecting a flat id list. Clearing it removes a pointless sort. No measurable change in benchmarking at 400k documents -- the id-list materialization/IN-clause cost still dominates -- but it's a free, strictly-correct cleanup, not just noise-neutral in the other direction. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>