mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-28 05:33:24 +00:00
docs(search): pin the stem-substitution limit wildcards inherit
Stemming substitutes as well as truncates ("copy" and "copies" both index as
"copi" while "copyright" keeps its literal y), so a stemmed pattern reaches a
word's inflections but no longer reaches compounds that keep the surface
spelling. That trade is accepted: the same substitution is what makes company*
and library* work, and no rule over one normalized string separates them. So
usage.md stops claiming a trailing star just works, and a test pins copy* to the
base word rather than the compound. Also adds a parity test tying
stem_pattern_text to paperless_text_analyzer's own output, so a filter added to
the index analyzer alone cannot silently diverge, and corrects the docstring
claim that a run can analyze to several tokens - the raw tokenizer emits one
token whatever the input, so only the remove_long zero-token case can fire.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5
parent
e311c84139
commit
e48182c358
+6
-4
@@ -927,10 +927,12 @@ title:Invoice*
|
||||
```
|
||||
|
||||
Wildcards are matched against the _stemmed_ terms stored in the index, not
|
||||
against the words as they appear in the document. A trailing `*` therefore
|
||||
works as expected (`invoice*` finds "invoice", "invoices" and "invoiced"), but
|
||||
a pattern whose text continues past where stemming cuts a word off cannot
|
||||
match: `productname` is indexed as `productnam`, so `produ*name` finds nothing.
|
||||
against the words as they appear in the document. A trailing `*` matches a word
|
||||
and its inflections (`invoice*` finds "invoice", "invoices" and "invoiced"),
|
||||
but not every longer word that starts with the same letters: `copy*` finds
|
||||
"copy" and "copies", not "copyright". For the same reason, a pattern whose text
|
||||
continues past where stemming cuts a word off cannot match at all:
|
||||
`productname` is indexed as `productnam`, so `produ*name` finds nothing.
|
||||
|
||||
Matching natural date keywords:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user