Simplify: dedupe file-swap rebuild, migration-check locking, and test mocks

Prompted by an automated simplification review of the branch. Applies the
highest-value findings, all behavior-preserving (full suite still green):

- vector_store.py: extract _rebuild_file() (temp-file lifecycle: open,
  populate, swap-in-or-discard-on-failure) and _rebuild_into() (create
  table, copy meta, stream rows) out of compact() and the v1->v2
  migration, which had duplicated both almost verbatim. The migration
  module shrinks to a single _rebuild_into() call instead of reaching
  into four PaperlessSqliteVecVectorStore privates.
- vector_store.py: extract _stored_schema_version() out of
  has_pending_migration() and check_and_run_migrations(), which
  duplicated the same schema_version read and its "missing key means
  current" default.
- indexing.py: extract _with_exclusive_access(), collapsing three
  identical _exclude_readers()/Timeout/log-and-skip blocks (compaction
  in update_llm_index() and llm_index_compact(), the migration check in
  _check_and_run_migrations()) to one line each. Also trims
  _check_and_run_migrations()'s docstring, which had grown to restate
  content already documented on has_pending_migration() and
  check_and_run_migrations(), including a claim that was now stale
  (llm_index_migrate() is a second consumer of the re-embed signal).
- test_ai_indexing.py: add a mock_store fixture for the
  write_store()-yields-a-MagicMock pattern that 8 tests were hand-rolling
  identically.
- migrations/__init__.py: consolidate the "how to add a migration"
  instructions to one place instead of three.
- docs/administration.md: fix two now-inaccurate claims -- the bare-metal
  migrate step doesn't run "automatically" (it's the manual step being
  documented), and the self-contradictory "if enabled... no-op if
  disabled" phrasing on the same step.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
stumpylog
2026-07-28 12:32:59 -07:00
co-authored by Claude Sonnet 5
parent 470bcc1751
commit 15938945d7
6 changed files with 174 additions and 206 deletions
+9 -10
View File
@@ -212,16 +212,15 @@ following:
This is a no-op if the index is already up to date, so it is safe to
run on every upgrade.
5. If the LLM index is enabled, apply any pending LLM index schema
migrations.
5. Apply any pending LLM index schema migrations.
```shell-session
cd src
python3 manage.py document_llmindex migrate
```
This is a no-op if the index is already up to date (or the LLM index
is disabled), so it is safe to run on every upgrade.
This is a no-op if the index is already up to date, or if the LLM index
is disabled, so it is safe to run on every upgrade.
### Database Upgrades
@@ -557,12 +556,12 @@ Specify `compact` to reclaim space and optimize the on-disk vector store.
Specify `migrate` to apply any pending index schema migrations without a full reindex.
This is a no-op if the index is already up to date, so it is safe to run on every
startup or upgrade; it is what the container's startup sequence and the
[bare-metal upgrade steps](#bare-metal-updating) run automatically. If a pending
migration would require re-embedding every document, `migrate` only logs a warning and
leaves the index as-is -- re-embedding can be slow and, for a metered embedding backend,
cost money, so it is never triggered automatically. Run `rebuild` yourself when you are
ready.
startup or upgrade; the container's startup sequence runs it automatically, and the
[bare-metal upgrade steps](#bare-metal-updating) include it as a manual step. If a
pending migration would require re-embedding every document, `migrate` only logs a
warning and leaves the index as-is -- re-embedding can be slow and, for a metered
embedding backend, cost money, so it is never triggered automatically. Run `rebuild`
yourself when you are ready.
!!! note