amorfo77/paperless-ngx
1
0
Fork 0
mirror of https://github.com/paperless-ngx/paperless-ngx.git synced 2026-08-12 05:43:18 +00:00
Code Issues Projects Releases Wiki Activity

chore: update transition guidance for the current whoosh-compat API

Browse Source 
Field references became a typed value rather than a dotted string, so
diagnostics carry one too and the registry exposes a single resolver.
Also records that the JSON fields must stay non-fast while existence
checks against a fast JSON field return inverted results.
This commit is contained in:
stumpylog
2026-08-11 14:21:52 -07:00
parent c86edaa0e8
commit 812e784a68
1 changed files with 7 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/whoosh-compat-transition/SKILL.md
+7 -1
View File
@@ -15,7 +15,9 @@ whoosh-compat (github.com/stumpylog/whoosh-compat; local checkout usually at `..
- **The field surface is a policy decision, not `KNOWN_FIELDS`.** Today's `KNOWN_FIELDS` accepts internal ID fields (`tag_id`, `owner_id`, `viewer_id`, other `*_id`) that are undocumented in `docs/usage.md` and were ruled not user-searchable by the maintainer: exclude them from the `FieldRegistry` (they stay as programmatic permission/filter fields in `build_permission_filter`, which never touches user query text). The registry is built from documented syntax in `docs/usage.md` plus the v2-compat aliases (`type`, `path`, `type_id`-style aliases follow their canonical field's fate). Undocumented-but-working fields (`asn`, `page_count`, `num_notes`, `original_filename`, `checksum`) need an explicit maintainer yes/no; since users type freely, silently dropping one breaks any saved view using it, so a drop must be a visible, documented decision.
- **Analyzer seam:** `FieldSpec.analyzer` binds the live registered tantivy analyzer's `.analyze` (the same Rust analyzer used at index time; language-keyed, so rebuild the registry when `SEARCH_LANGUAGE` changes, on the same trigger as `register_tokenizers`). `pattern_normalizer` is `_tokenizer.ascii_fold`: character-level lowercase+fold only, NEVER stemming.
- **Diagnostics before emit:** `whoosh_compat.parse()` never raises on bad input. Check `ParseResult.diagnostics` and map to `SearchQueryError`/`InvalidDateQuery` (HTTP 400) BEFORE calling `emit()`; also catch the emitter's `UnsupportedQueryError` into a 400. Never carry forward the legacy raw-string fallback (`except Exception: query_str = raw_query`) into the new path; it masks integration bugs.
- **Build typed errors from structured diagnostic data, never by parsing `message`.** Each `Diagnostic` carries `kind`, `startchar`/`endchar`, and `field`/`raw_value`. `InvalidDateQuery(field, value)` and any `InvalidNumberQuery` come straight from `diagnostic.field` and `diagnostic.raw_value`; `message` is human-readable text whose wording can change. For a range that fails on one bound, `raw_value` is the bound that actually failed.
- **Build typed errors from structured diagnostic data, never by parsing `message`.** Each `Diagnostic` carries `kind`, `startchar`/`endchar`, and `field`/`raw_value`. `message` is human-readable text whose wording can change. For a range that fails on one bound, `raw_value` is the bound that actually failed.
- `diagnostic.field` is a **`FieldRef`**, not a string: use `str(diagnostic.field)` for the canonical dotted name (`created`, `notes.user`) or `diagnostic.field.name` for the field alone. Note the name is canonical, so an aliased query (`type:`) reports the field it resolves to (`document_type`), and the diagnostic span covers the offending value rather than the field name, so the text the user typed for the field is not recoverable.
- **The registry has one resolver.** `registry.make_ref(raw)` turns a raw field string into a `FieldRef` or `None` for an unknown field, and `registry.resolve(ref)` returns the spec. There is no `resolve_json()`; a dotted name is interpreted only inside `make_ref`.
- **`notes` and `custom_fields` are JSON fields** with fixed subpaths (`notes.user`/`notes.note`, `custom_fields.name`/`custom_fields.value`); the registry stays a static, language-keyed singleton, never per-request.
## Mandatory before deleting old code
@@ -40,6 +42,10 @@ Added:
- One `Multitoken` case nested inside a top-level `OR` (whoosh-compat DIVERGENCES entry on Multitoken.DEFAULT) to prove it does not matter for paperless's data.
- If acceptance work surfaces a new whoosh-compat divergence, that is a whoosh-compat-repo change (its `differential-triage` skill applies), not a silent paperless workaround.
## Do not mark the JSON fields fast
`notes` and `custom_fields` must stay non-fast for now. Existence checks against a **fast** JSON field currently return inverted results (a document that has the field is reported as not having it), because the underlying call does not check subpath columns by default. The unsupported-configuration error raised for a non-fast JSON field advises marking it fast, which walks directly into that bug. Ignore that advice until the upstream issue is fixed, then re-check.
## Coordination
- whoosh-compat is pre-1.0: pin an exact version or git SHA; upgrades are deliberate, reviewed changes.