mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-01 00:22:17 +00:00
Simplify: dedupe file-swap rebuild, migration-check locking, and test mocks
Prompted by an automated simplification review of the branch. Applies the highest-value findings, all behavior-preserving (full suite still green): - vector_store.py: extract _rebuild_file() (temp-file lifecycle: open, populate, swap-in-or-discard-on-failure) and _rebuild_into() (create table, copy meta, stream rows) out of compact() and the v1->v2 migration, which had duplicated both almost verbatim. The migration module shrinks to a single _rebuild_into() call instead of reaching into four PaperlessSqliteVecVectorStore privates. - vector_store.py: extract _stored_schema_version() out of has_pending_migration() and check_and_run_migrations(), which duplicated the same schema_version read and its "missing key means current" default. - indexing.py: extract _with_exclusive_access(), collapsing three identical _exclude_readers()/Timeout/log-and-skip blocks (compaction in update_llm_index() and llm_index_compact(), the migration check in _check_and_run_migrations()) to one line each. Also trims _check_and_run_migrations()'s docstring, which had grown to restate content already documented on has_pending_migration() and check_and_run_migrations(), including a claim that was now stale (llm_index_migrate() is a second consumer of the re-embed signal). - test_ai_indexing.py: add a mock_store fixture for the write_store()-yields-a-MagicMock pattern that 8 tests were hand-rolling identically. - migrations/__init__.py: consolidate the "how to add a migration" instructions to one place instead of three. - docs/administration.md: fix two now-inaccurate claims -- the bare-metal migrate step doesn't run "automatically" (it's the manual step being documented), and the self-contradictory "if enabled... no-op if disabled" phrasing on the same step. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 5
parent
470bcc1751
commit
15938945d7
+9
-10
@@ -212,16 +212,15 @@ following:
|
||||
This is a no-op if the index is already up to date, so it is safe to
|
||||
run on every upgrade.
|
||||
|
||||
5. If the LLM index is enabled, apply any pending LLM index schema
|
||||
migrations.
|
||||
5. Apply any pending LLM index schema migrations.
|
||||
|
||||
```shell-session
|
||||
cd src
|
||||
python3 manage.py document_llmindex migrate
|
||||
```
|
||||
|
||||
This is a no-op if the index is already up to date (or the LLM index
|
||||
is disabled), so it is safe to run on every upgrade.
|
||||
This is a no-op if the index is already up to date, or if the LLM index
|
||||
is disabled, so it is safe to run on every upgrade.
|
||||
|
||||
### Database Upgrades
|
||||
|
||||
@@ -557,12 +556,12 @@ Specify `compact` to reclaim space and optimize the on-disk vector store.
|
||||
|
||||
Specify `migrate` to apply any pending index schema migrations without a full reindex.
|
||||
This is a no-op if the index is already up to date, so it is safe to run on every
|
||||
startup or upgrade; it is what the container's startup sequence and the
|
||||
[bare-metal upgrade steps](#bare-metal-updating) run automatically. If a pending
|
||||
migration would require re-embedding every document, `migrate` only logs a warning and
|
||||
leaves the index as-is -- re-embedding can be slow and, for a metered embedding backend,
|
||||
cost money, so it is never triggered automatically. Run `rebuild` yourself when you are
|
||||
ready.
|
||||
startup or upgrade; the container's startup sequence runs it automatically, and the
|
||||
[bare-metal upgrade steps](#bare-metal-updating) include it as a manual step. If a
|
||||
pending migration would require re-embedding every document, `migrate` only logs a
|
||||
warning and leaves the index as-is -- re-embedding can be slow and, for a metered
|
||||
embedding backend, cost money, so it is never triggered automatically. Run `rebuild`
|
||||
yourself when you are ready.
|
||||
|
||||
!!! note
|
||||
|
||||
|
||||
Reference in New Issue
Block a user