mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-01 16:42:18 +00:00
02c6b5f3e4
DocumentViewSet.get_queryset() carried a blanket .distinct() left over from an older query shape (predating the version-file feature and the correlated-subquery rewrite of num_notes/effective_content). Nothing in the current base queryset can produce duplicate document rows: select_related is all FK-to-PK, and the permission filter is a boolean id__in predicate, not a join. id (the PK) is also always selected and inherently unique, so .distinct() was a structural no-op for correctness. It was not a no-op for cost. EXPLAIN showed it forcing a full sort-then- dedupe over the entire permission-visible document set before LIMIT could apply -- 340k rows for a 25-row page, with the effective_content/num_notes correlated subqueries re-executed once per row as a result. This was the dominant cost behind the original report's document overview slowness, well beyond the get_user_can_change N+1 fixed separately. Removing it isn't safe on its own, though -- auditing found two existing filters that rely on an M2M join *without* deduping themselves, previously papered over by the blanket .distinct(): - InboxFilter: a document with two tags both flagged is_inbox_tag=True (nothing prevents that) would be returned twice for ?is_in_inbox=true. - CustomFieldsFilter: a document with multiple custom field instances matching different OR-ed branches would be returned once per match. Fixed both locally, matching the pattern ObjectFilter already used for tags__id__in/custom_fields__id__in. Added regression tests for both -- confirmed they fail without the local .distinct() calls. Benchmarked against the same 400k-document/1,000-tag corpus: document list wall-clock for a permission-restricted user drops from ~15.8s to ~2.25s (~7x), closing most of the previous gap to a superuser's ~1.8s. Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>