mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-04-07 00:28:54 +00:00
Removes the profiling data
This commit is contained in:
Trenton H
File diff suppressed because it is too large
Load Diff
@@ -1,78 +0,0 @@
|
||||
Profiling data after Option 1: search_ids + page-only highlight_hits
|
||||
=====================================================================
|
||||
Run date: 2026-04-05
|
||||
Commit: 610ba2789 (feat: replace 10000 overfetch with search_ids + page-only highlights)
|
||||
Test corpus: 200 documents
|
||||
|
||||
============================================================
|
||||
Profile: relevance search (no ordering)
|
||||
============================================================
|
||||
Wall time: 0.9167s
|
||||
Queries: 33 (0.0000s)
|
||||
Memory delta: 16477.8 KiB
|
||||
Peak memory: 16504.5 KiB
|
||||
|
||||
============================================================
|
||||
Profile: sorted search (ordering=created)
|
||||
============================================================
|
||||
Wall time: 0.1378s
|
||||
Queries: 32 (0.0000s)
|
||||
Memory delta: 792.1 KiB
|
||||
Peak memory: 818.9 KiB
|
||||
|
||||
============================================================
|
||||
Profile: paginated search (page=2, page_size=25)
|
||||
============================================================
|
||||
Wall time: 0.1322s
|
||||
Queries: 32 (0.0000s)
|
||||
Memory delta: 788.3 KiB
|
||||
Peak memory: 815.2 KiB
|
||||
|
||||
============================================================
|
||||
Profile: search with selection_data
|
||||
============================================================
|
||||
Wall time: 0.1570s
|
||||
Queries: 37 (0.0010s)
|
||||
Memory delta: 837.3 KiB
|
||||
Peak memory: 981.2 KiB
|
||||
|
||||
============================================================
|
||||
Profile: backend.search(page_size=10000, all highlights)
|
||||
============================================================
|
||||
Wall time: 0.0193s
|
||||
Queries: 0 (0.0000s)
|
||||
Memory delta: 88.6 KiB
|
||||
Peak memory: 100.2 KiB
|
||||
|
||||
============================================================
|
||||
Profile: backend.search(page_size=25)
|
||||
============================================================
|
||||
Wall time: 0.0046s
|
||||
Queries: 0 (0.0000s)
|
||||
Memory delta: 5.9 KiB
|
||||
Peak memory: 11.1 KiB
|
||||
|
||||
|
||||
Comparison summary (200 docs):
|
||||
==============================
|
||||
|
||||
| Scenario | Baseline | After Option 1 | Change |
|
||||
|---------------------------|-----------|----------------|--------------|
|
||||
| Relevance — wall | 0.962s | 0.917s | -5% |
|
||||
| Relevance — memory | 16557 KiB | 16478 KiB | -0.5% |
|
||||
| Sorted — wall | 0.132s | 0.138s | ~same |
|
||||
| Sorted — memory | 881 KiB | 792 KiB | -10% |
|
||||
| Paginated — wall | 0.140s | 0.132s | -6% |
|
||||
| Paginated — memory | 868 KiB | 788 KiB | -9% |
|
||||
| Selection data — wall | 0.166s | 0.157s | -5% |
|
||||
| Selection data — memory | 927 KiB | 837 KiB | -10% |
|
||||
| Backend 10k — wall | 0.018s | 0.019s | same |
|
||||
| Backend 10k — memory | 89 KiB | 89 KiB | same |
|
||||
| Backend 25 — wall | 0.007s | 0.005s | -29% |
|
||||
| Backend 25 — memory | 5.9 KiB | 5.9 KiB | same |
|
||||
|
||||
Notes:
|
||||
- The 10000 hardcoded limit has been removed; search_ids() now returns all matches.
|
||||
- Relevance search is dominated by first-request import overhead (~16 MiB).
|
||||
- Memory savings will scale with document count (ints vs SearchHit dicts).
|
||||
- Backend-only 10k test is unchanged because it still calls search() directly.
|
||||
@@ -1,121 +0,0 @@
|
||||
============================= test session starts ==============================
|
||||
platform linux -- Python 3.14.3, pytest-9.0.2, pluggy-1.6.0 -- /home/trenton/Documents/projects/paperless-ngx/.venv/bin/python
|
||||
cachedir: .pytest_cache
|
||||
django: version: 5.2.12, settings: paperless.settings (from ini)
|
||||
rootdir: /home/trenton/Documents/projects/paperless-ngx
|
||||
configfile: pyproject.toml
|
||||
plugins: sugar-1.1.1, xdist-3.8.0, cov-7.0.0, httpx-0.36.0, django-4.12.0, Faker-40.8.0, env-1.5.0, time-machine-3.2.0, mock-3.15.1, anyio-4.12.1, rerunfailures-16.1
|
||||
collecting ... collected 6 items
|
||||
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_relevance_search Creating test database for alias 'default'...
|
||||
|
||||
============================================================
|
||||
Profile: BEFORE — relevance search (no ordering)
|
||||
============================================================
|
||||
Wall time: 0.9622s
|
||||
Queries: 33 (0.0000s)
|
||||
Memory delta: 16557.2 KiB
|
||||
Peak memory: 16584.0 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
<frozen importlib._bootstrap_external>:511: size=5480 KiB (+5480 KiB), count=45642 (+45642), average=123 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/fido2/rpid.py:47: size=518 KiB (+518 KiB), count=9769 (+9769), average=54 B
|
||||
<frozen abc>:106: size=432 KiB (+432 KiB), count=1480 (+1480), average=299 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/langdetect/utils/ngram.py:257: size=391 KiB (+391 KiB), count=6667 (+6667), average=60 B
|
||||
<frozen importlib._bootstrap>:491: size=284 KiB (+284 KiB), count=2543 (+2543), average=114 B
|
||||
============================================================
|
||||
|
||||
PASSED
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_sorted_search
|
||||
============================================================
|
||||
Profile: BEFORE — sorted search (ordering=created)
|
||||
============================================================
|
||||
Wall time: 0.1320s
|
||||
Queries: 32 (0.0010s)
|
||||
Memory delta: 880.8 KiB
|
||||
Peak memory: 906.8 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:575: size=50.1 KiB (+50.1 KiB), count=521 (+521), average=99 B
|
||||
/home/trenton/.local/share/uv/python/cpython-3.14.3-linux-x86_64-gnu/lib/python3.14/copyreg.py:104: size=49.7 KiB (+49.7 KiB), count=315 (+315), average=162 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django/db/models/sql/query.py:386: size=38.0 KiB (+38.0 KiB), count=160 (+160), average=243 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django_filters/filterset.py:209: size=32.0 KiB (+32.0 KiB), count=82 (+82), average=400 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django_filters/filters.py:158: size=21.4 KiB (+21.4 KiB), count=104 (+104), average=210 B
|
||||
============================================================
|
||||
|
||||
PASSED
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_paginated_search
|
||||
============================================================
|
||||
Profile: BEFORE — paginated search (page=2, page_size=25)
|
||||
============================================================
|
||||
Wall time: 0.1395s
|
||||
Queries: 32 (0.0000s)
|
||||
Memory delta: 868.1 KiB
|
||||
Peak memory: 893.5 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:575: size=50.1 KiB (+50.1 KiB), count=521 (+521), average=99 B
|
||||
/home/trenton/.local/share/uv/python/cpython-3.14.3-linux-x86_64-gnu/lib/python3.14/copyreg.py:104: size=49.2 KiB (+49.2 KiB), count=315 (+315), average=160 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django/db/models/sql/query.py:386: size=38.1 KiB (+38.1 KiB), count=161 (+161), average=242 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django_filters/filterset.py:209: size=32.0 KiB (+32.0 KiB), count=82 (+82), average=400 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django_filters/filters.py:158: size=21.3 KiB (+21.3 KiB), count=104 (+104), average=209 B
|
||||
============================================================
|
||||
|
||||
PASSED
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_search_with_selection_data
|
||||
============================================================
|
||||
Profile: BEFORE — search with selection_data
|
||||
============================================================
|
||||
Wall time: 0.1656s
|
||||
Queries: 37 (0.0020s)
|
||||
Memory delta: 926.9 KiB
|
||||
Peak memory: 1084.3 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:575: size=50.1 KiB (+50.1 KiB), count=521 (+521), average=99 B
|
||||
/home/trenton/.local/share/uv/python/cpython-3.14.3-linux-x86_64-gnu/lib/python3.14/copyreg.py:104: size=49.6 KiB (+49.6 KiB), count=327 (+327), average=155 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django/db/models/sql/query.py:386: size=38.1 KiB (+38.1 KiB), count=161 (+161), average=242 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django_filters/filterset.py:209: size=32.0 KiB (+32.0 KiB), count=82 (+82), average=400 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/.venv/lib/python3.14/site-packages/django/db/backends/sqlite3/operations.py:193: size=27.1 KiB (+27.1 KiB), count=37 (+37), average=751 B
|
||||
============================================================
|
||||
|
||||
PASSED
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_backend_search_only
|
||||
============================================================
|
||||
Profile: BEFORE — backend.search(page_size=10000, all highlights)
|
||||
============================================================
|
||||
Wall time: 0.0175s
|
||||
Queries: 0 (0.0000s)
|
||||
Memory delta: 88.6 KiB
|
||||
Peak memory: 100.3 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:575: size=51.2 KiB (+51.2 KiB), count=530 (+530), average=99 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:557: size=17.8 KiB (+17.8 KiB), count=200 (+200), average=91 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:542: size=8576 B (+8576 B), count=134 (+134), average=64 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:522: size=4800 B (+4800 B), count=200 (+200), average=24 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:515: size=2376 B (+2376 B), count=99 (+99), average=24 B
|
||||
============================================================
|
||||
|
||||
PASSED
|
||||
src/documents/tests/test_search_profiling.py::TestSearchProfilingBaseline::test_profile_backend_search_single_page
|
||||
============================================================
|
||||
Profile: BEFORE — backend.search(page_size=25)
|
||||
============================================================
|
||||
Wall time: 0.0070s
|
||||
Queries: 0 (0.0000s)
|
||||
Memory delta: 5.9 KiB
|
||||
Peak memory: 11.3 KiB
|
||||
|
||||
Top 5 allocations:
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:557: size=2275 B (+2275 B), count=25 (+25), average=91 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:575: size=1600 B (+1600 B), count=25 (+25), average=64 B
|
||||
/home/trenton/.local/share/uv/python/cpython-3.14.3-linux-x86_64-gnu/lib/python3.14/weakref.py:73: size=1280 B (+1280 B), count=20 (+20), average=64 B
|
||||
/home/trenton/Documents/projects/paperless-ngx/src/documents/search/_backend.py:574: size=256 B (+256 B), count=1 (+1), average=256 B
|
||||
/home/trenton/.local/share/uv/python/cpython-3.14.3-linux-x86_64-gnu/lib/python3.14/tracemalloc.py:560: size=240 B (+240 B), count=1 (+1), average=240 B
|
||||
============================================================
|
||||
|
||||
PASSEDDestroying test database for alias 'default'...
|
||||
|
||||
|
||||
======================== 6 passed in 241.83s (0:04:01) =========================
|
||||
Reference in New Issue
Block a user