amorfo77/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2026-08-28 05:33:24 +00:00
Code Issues Projects Releases Wiki Activity

fix(search): keep wildcard patterns literal on unstemmed KEYWORD fields

Browse Source 
get_field_registry() branched the analyzer by field kind but gave every field
the same stemming pattern normalizer. checksum is indexed with the raw
tokenizer, so its terms are neither folded nor stemmed, yet its wildcard
patterns were: "checksum:ceded*" normalized to "cede*" and matched a document
whose checksum starts with "cedef00d". About 2.8% of random hex prefixes were
rewritten this way. Always over-matching rather than missing, but for a field
whose whole purpose is exact identification, returning a different checksum is
a wrong answer.

KEYWORD fields now get a fold-and-lower normalizer, which is what every field
used before pattern stemming was added; TEXT fields keep the stemming one so
"invoice*" still reaches the indexed "invoic".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
stumpylog
2026-08-20 12:08:47 -07:00
co-authored by Claude Opus 5
parent 0b080d9415
commit 82546f13b3
3 changed files with 102 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/usage.md
+1 -1
View File
@@ -933,7 +933,7 @@ original_filename:invoice.pdf
- `asn` matches a document's Archive Serial Number.
- `page_count` matches a document's page count.
- `num_notes` matches how many notes a document has.
- `checksum` matches the checksum of the original document file (not the archived/processed version). Unlike the text fields, this one is stored verbatim rather than tokenized, so only a complete, lowercase checksum matches. To search by the first few characters instead, use a wildcard: `checksum:9f86d081*`. Because the field is not stemmed, that prefix is matched literally.
- `checksum` matches the checksum of the original document file (not the archived/processed version). Unlike the text fields, this one is stored verbatim rather than tokenized, so only a complete, lowercase checksum matches. To search by the first few characters instead, use a wildcard: `checksum:9f86d081*`. Wildcard patterns on the text fields are stemmed to line up with the stemmed index, but `checksum` is indexed without stemming, so its patterns are not stemmed either and the prefix is matched exactly as typed.
- `original_filename` matches the filename of the document as originally consumed.
`asn`, `page_count` and `num_notes` are numeric and also accept ranges, for example `asn:[50 to 150]`.