mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-08-27 05:03:20 +00:00
stumpylogandClaude Opus 5
e48182c358
Stemming substitutes as well as truncates ("copy" and "copies" both index as
"copi" while "copyright" keeps its literal y), so a stemmed pattern reaches a
word's inflections but no longer reaches compounds that keep the surface
spelling. That trade is accepted: the same substitution is what makes company*
and library* work, and no rule over one normalized string separates them. So
usage.md stops claiming a trailing star just works, and a test pins copy* to the
base word rather than the compound. Also adds a parity test tying
stem_pattern_text to paperless_text_analyzer's own output, so a filter added to
the index analyzer alone cannot silently diverge, and corrects the docstring
claim that a run can analyze to several tokens - the raw tokenizer emits one
token whatever the input, so only the remove_long zero-token case can fire.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>