amorfo77/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2026-08-21 02:03:30 +00:00
Code Issues Projects Releases Wiki Activity

fix: skip fuzzy search blend when raw query isn't tantivy-parseable

Browse Source 
The fuzzy blend clause in parse_user_query() fed the raw, whoosh-syntax
query string directly to tantivy's own query parser. Since the
whoosh-compat migration, raw_query still contains whoosh grammar (date
keywords, whoosh-style ranges, bracket-class wildcards) that tantivy's
parser rejects with ValueError, which escaped parse_user_query and
turned into a generic HTTP 400 for the entire query whenever
ADVANCED_FUZZY_SEARCH_THRESHOLD was configured.

Deriving a clean plain-text-only extraction for the fuzzy clause was
ruled out: wc.parse() already expands unfielded terms into per-default-
field copies in the AST, so there's no "still unfielded" marker left to
walk without duplicating whoosh-compat's own expansion logic. Instead,
scope a narrow try/except ValueError around exactly the
index.parse_query() call and skip the fuzzy clause (logged at debug)
when it can't parse, leaving the exact/CJK clauses unaffected.
This commit is contained in:
Trenton Holmes
2026-08-19 13:36:53 -07:00
committed by stumpylog
parent 5941c19fb4
commit f5e7d309af
2 changed files with 61 additions and 10 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/documents/tests/search/test_query.py
+16 -1
View File
@@ -36,13 +36,28 @@ class TestParseUserQuery:
def test_returns_tantivy_query(self, query_index: tantivy.Index) -> None:
assert isinstance(parse_user_query(query_index, "invoice", UTC), tantivy.Query)
@pytest.mark.parametrize(
"raw_query",
[
pytest.param("invoice", id="plain_text"),
pytest.param("created:today", id="date_keyword"),
pytest.param("created:[2005 to 2009]", id="whoosh_date_range"),
pytest.param('added:"previous month"', id="quoted_date_phrase"),
pytest.param("title:202[0-1]*", id="bracket_class_wildcard"),
],
)
def test_fuzzy_mode_does_not_raise(
self,
query_index: tantivy.Index,
settings,
raw_query: str,
) -> None:
# These are all valid whoosh grammar that tantivy's own query parser
# (used only by the fuzzy blend clause) cannot parse; the fuzzy
# clause must degrade gracefully instead of raising and failing the
# whole query. See _try_parse_fuzzy_query.
settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.5
assert isinstance(parse_user_query(query_index, "invoice", UTC), tantivy.Query)
assert isinstance(parse_user_query(query_index, raw_query, UTC), tantivy.Query)
def test_date_rewriting_applied_before_tantivy_parse(
self,