mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-20 17:53:20 +00:00
Trenton HolmesandClaude Fable 5
8789ff55f5
The fuzzy blend clause handed the raw query string to tantivy's own parser, which rejects whoosh-only grammar (date keywords, whoosh ranges, aliases needing resolution), so any mixed query silently lost its fuzzy clause: a typo'd word beside "added:today" stopped matching the moment the date keyword appeared, while the same typo without it still matched. Before the whoosh-compat migration the parser received the translated string, so fuzzy survived mixed queries. The clause is now built from whoosh_compat.free_text_tokens over the already-parsed AST: the query's free-text words, analyzed, deduplicated, with negated terms excluded so a NOT'd word cannot resurface through the fuzzy clause. The joined word string is always plain tokens, so tantivy always parses it; a defensive word-character filter guards any future field whose analyzer passes punctuation through, and the ValueError skip remains as insurance. One chosen trade-off is documented in the docstring: a term fielded on a default search field contributes its text unfielded, widening fuzzy recall on the 0.1-boosted secondary clause. Two result-level acceptance tests pin the behavior: the mixed typo-plus-date-keyword query matches its document again, and a NOT'd word does not fuzzy-resurface (shaped so the assertion genuinely fails under a naive all-words implementation: the excluded word's document is the only candidate hit, so score normalization cannot mask it). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WMsn6DgzbvSqh1pwy66VVF