Commit Graph
1 Commits
Author SHA1 Message Date
stumpylogandClaude Opus 5 4e2d71513a fix(search): build the CJK clause from the parsed AST, not the raw query
_build_cjk_query scanned the raw query string for CJK runs, so a CJK term
the user negated ('invoice NOT 漢字') or restricted to one field
('title:漢字', 'notes:漢字') came straight back as a top-level Should
clause over every bigram field. The fuzzy clause already collects its
words from the parsed tree for exactly this reason; the CJK clause a few
lines below did not.

Collect the CJK runs from whoosh_compat's free_text_tokens over
result.ast instead, one default field at a time so the tokens keep their
field attribution: a bare term (already copied onto every default field
by the parser) still searches every bigram field, while title:東京
reaches bigram_title alone, and a term on a non-default field
contributes nothing. Fields sharing identical CJK text share one parse.

The raw-string builder stays for the simple TEXT/TITLE modes, whose
input is plain text with no query grammar to respect, as does
extract_cjk_text, which the indexing side calls per bigram field.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-20 08:44:47 -07:00