mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-07-28 14:54:55 +00:00
84a04301d1
* Fix (beta): tag/custom-field document_count scales badly with tag count TagViewSet and CustomFieldViewSet's document_count annotation used a per-row correlated subquery (annotate_document_count_for_related_queryset), executed once per tag/custom-field row. Fine at a few dozen rows, but at ~1,000 tags it degrades to a full per-tag GroupAggregate over the tags M2M table -- 6s+ in production reports, confirmed via EXPLAIN (loops=1000). Replaced with a single, independent GROUP BY over the through table (permission filter expressed as a plain WHERE, not an aggregate FILTER), then injected via Case/When. Also tried a more "obvious" fix -- a plain Count(filter=Q(id__in=permitted_ids), distinct=True) directly on the M2M relation -- but that's worse: Postgres fails to plan the id__in check as a semi-join once a large M2M bridge table is involved, and instead re-checks subquery membership once per joined row. Benchmarked against a synthetic corpus (400k documents, 1,000 tags @ ~5/doc, matching real-world reports in discussion #13161): tag list wall-clock drops from ~180s to ~8s for a permission-restricted user, ~21s to ~8s for a superuser. No measurable change at typical home-instance scale (tens of tags). Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> * Address review feedback: restore validation, scope aggregation to queryset - Restore the document_count_source_field validation helper dropped during the refactor -- misconfiguring a viewset (document_count_through set without document_count_source_field) now fails fast with a clear error again instead of an obscure runtime failure. - Restrict annotate_document_count_for_related_queryset()'s through-table aggregation to rows whose related_object_field is one of the annotated queryset's pks. No-op for the main tag-list call site (queryset is all tags), but avoids unnecessary work for narrower callers like the tag descendants branch in TagViewSet.list(). Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>