When open_or_rebuild_index is called and the index directory does not exist,
return a fresh in-memory Tantivy index instead of creating the directory as
a side effect. This prevents workspace contamination during test runs where
INDEX_DIR has not been redirected to a temp directory.
In production the data directory is always created during setup, so disk-
based indexes continue to work normally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- `open_or_rebuild_index` now calls `index_dir.mkdir(parents=True, exist_ok=True)`
so a missing index directory is created on demand rather than crashing on
`iterdir()` inside `wipe_index`
- `TestTagHierarchy.setUp` calls `super().setUp()` so `DirectoriesMixin` runs
and `self.dirs` is set before teardown tries to clean up
- `test_search_more_like` d4 content changed to words with no overlap with d2/d3
to avoid spurious MLT hits from shared stop words at `min_doc_frequency=1`
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add comprehensive docstrings to all public methods and classes in the search package
- Clarify purpose, parameters, return values, and implementation notes
- Document thread safety, error handling, and usage patterns
- Explain Tantivy-specific workarounds and design decisions
- Improve test quality and pytest compliance
- Add descriptive comments explaining what each test verifies
- Convert TestIndexOptimize to pytest style with @pytest.mark.django_db
- Ensure all test docstrings focus on behavior verification rather than implementation
- Maintain existing functionality while improving code documentation
- No changes to production logic or test coverage
- All tests continue to pass with enhanced clarity
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename _needs_rebuild -> needs_rebuild and export from documents.search
- document_index command imports directly from documents.search, constructs
the queryset and calls get_backend().rebuild() inline — no tasks.py indirection
- Optimize subcommand logs deprecation directly; no longer calls index_optimize
- Remove index_reindex from tasks.py
- Convert TestMakeIndex to pytest class (no TestCase); use mocker fixtures
- Simplify TestIndexReindex -> TestIndexOptimize (wrapper test removed)
Co-Authored-By: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SEARCH_LANGUAGE is now str | None (None = no stemming, not "")
- When PAPERLESS_SEARCH_LANGUAGE is set, validate it against
SUPPORTED_LANGUAGES via get_choice_from_env (startup error on bad value)
- When not set, infer from OCR_LANGUAGE's primary Tesseract code
(eng→en, deu→de, fra→fr, etc.) covering all 18 Tantivy-supported languages
- _schema.py sentinel normalises None → "" for on-disk comparison
- _tokenizer.py type annotations updated to str | None
- docs: recommend ISO 639-1 two-letter codes; note that capitalized
Tantivy enum names are not valid; link to Tantivy Language enum
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TantivyBackend now uses open()/close()/_ensure_open() instead of __enter__/__exit__.
get_backend() tracks _backend_path and auto-reinitializes when settings.INDEX_DIR
changes, fixing the xdist/override_settings isolation bug where parallel workers
would share a stale singleton pointing at a deleted index directory.
Test fixtures use in-memory indices (path=None) for speed and isolation.
Singleton behavior covered by TestSingleton in test_backend.py.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace all `from documents import index` + Whoosh writer usage across
admin.py, bulk_edit.py, tasks.py, views.py, signals/handlers.py with
`get_backend().add_or_update/remove/batch_update`
- Add `effective_content` param to `_build_tantivy_doc` / `add_or_update`
(used by signal handler to re-index root doc with version's OCR text)
- Add `wipe_index()` (renamed from `_wipe_index`) to public API; use from
`document_index --recreate` flag
- `index_optimize()` replaced with deprecation log message; Tantivy
manages segment merging automatically
- `index_reindex()` now calls `get_backend().rebuild()` + `reset_backend()`
with select_related/prefetch_related for efficiency
- `document_index` management command: add `--recreate` flag
- Status view: use `get_backend()` + dir mtime scan instead of Whoosh
`ix.last_modified()`
- Delete `documents/index.py`, `test_index.py`, `test_delayedquery.py`
- Update all tests: patch `documents.search.get_backend` (lazy imports);
`DirectoriesMixin` calls `reset_backend()` in setUp/tearDown;
`TestDocumentConsumptionFinishedSignal` likewise
- `test_api_search.py`: fix order-independent assertions for date-range
queries; fix `_rewrite_8digit_date` to be field-aware and
timezone-correct for DateTimeField vs DateField
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add descriptive docstrings to all functions in _schema.py, _tokenizer.py, and _query.py
- Complete type annotations for all function parameters and return values
- Fix 8 mypy strict errors in _query.py:
- Add re.Match[str] type parameters for regex matches
- Fix "Returning Any" error with str() cast
- Add type annotations for build_permission_filter() and parse_user_query()
- Remove lazy imports, move to module top level
- All 29 search module tests continue to pass
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implement date/timezone boundary math for natural language date queries:
- `created` (DateField): local calendar date to UTC midnight boundaries
- `added`/`modified` (DateTimeField): local day boundaries with full offset arithmetic
- Whoosh compat shims: compact dates (YYYYMMDDHHmmss) → ISO 8601
- Relative ranges: `[now-7d TO now]` → concrete ISO timestamps
- Natural keywords: today, yesterday, this_week, last_week, etc.
- Timezone-aware: handles UTC offset arithmetic for datetime fields
- Passthrough: bare keywords without field prefixes unchanged
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trenton HandClaude Sonnet 4.6
