The fuzzy blend clause handed the raw query string to tantivy's own
parser, which rejects whoosh-only grammar (date keywords, whoosh ranges,
aliases needing resolution), so any mixed query silently lost its fuzzy
clause: a typo'd word beside "added:today" stopped matching the moment
the date keyword appeared, while the same typo without it still matched.
Before the whoosh-compat migration the parser received the translated
string, so fuzzy survived mixed queries.
The clause is now built from whoosh_compat.free_text_tokens over the
already-parsed AST: the query's free-text words, analyzed, deduplicated,
with negated terms excluded so a NOT'd word cannot resurface through the
fuzzy clause. The joined word string is always plain tokens, so tantivy
always parses it; a defensive word-character filter guards any future
field whose analyzer passes punctuation through, and the ValueError skip
remains as insurance. One chosen trade-off is documented in the
docstring: a term fielded on a default search field contributes its text
unfielded, widening fuzzy recall on the 0.1-boosted secondary clause.
Two result-level acceptance tests pin the behavior: the mixed
typo-plus-date-keyword query matches its document again, and a NOT'd
word does not fuzzy-resurface (shaped so the assertion genuinely fails
under a naive all-words implementation: the excluded word's document is
the only candidate hit, so score normalization cannot mask it).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMsn6DgzbvSqh1pwy66VVF
A prior commit deleted test_query.py's parametrized "doesn't raise" coverage
for this shape (created:[...T...Z TO ...] and comma-combined ranges), which
was also the only place PR #13010's T/Z backward-compat guarantee was
exercised. Nothing in paperless's suite proved the full parse_user_query() ->
tantivy Query -> matched-document pipeline still honors it after the
whoosh-compat grammar fix (commit f936143 in the whoosh-compat repo). Add
result-level acceptance cases: an in/out-of-range T/Z bracket range, PR
#13010's original comma-combined two-field shape, and an exact-boundary case
proving a Z-suffixed bound is absolute UTC, not shifted by the local search
timezone.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RVj8NFy821G3YhNf68PF6X
test_unregistered_id_field_folds_to_literal_text_not_error only checked
that parse_user_query() didn't raise for a query like tag_id:5. Add a
result-level acceptance test (matching test_acceptance.py's
_matched_ids pattern, indexed against real documents) that asserts the
matched-document-ID set is genuinely empty, not just that the parse
step succeeds.
Replaces test_query.py's intermediate-AST/query-string checks with a
result-level acceptance corpus that indexes real documents and asserts
matched-ID sets through parse_user_query(), covering the #13568
bracket-wildcard regression, comma value lists, field boosts, JSON subpaths,
and Multitoken-in-OR nesting. Removes TestCreatedDateField, TestDateTimeFields,
TestWhooshQueryRewriting, TestYearRangeRewriting, TestNonDateFieldsNotRewritten,
TestPassthrough, TestNormalizeQuery, and TestParseUserQuery's
test_advanced_search_queries_do_not_raise from test_query.py, since they test
translate_query/_dates.py internals or a diagnostics-free-parse guarantee
whoosh-compat's own suite already covers.
Trenton HolmesandClaude Fable 5