docs: flag where the transition spec and plan describe a moved API

The library changed after these were written and more changes are already decided upstream. Records what is wrong today, what to write toward, the one question still open, and the fast-JSON-field trap, rather than silently leaving code that would fail on contact.
chore: update transition guidance for the current whoosh-compat API
2026-08-13 06:13:20 +00:00 · 2026-08-11 14:24:24 -07:00 · 2026-08-11 14:21:52 -07:00 · 2026-08-11 09:58:49 -07:00 · 2026-08-09 21:27:46 -07:00 · 2026-08-09 21:27:46 -07:00
44 changed files with 3018 additions and 2004 deletions
@@ -0,0 +1,62 @@
+---
+name: whoosh-compat-transition
+description: Use when integrating the whoosh-compat library into paperless-ngx search, replacing src/documents/search/_translate.py or _dates.py, building the search FieldRegistry, or changing user query parsing during the whoosh-to-tantivy transition
+---
+
+# whoosh-compat transition
+
+## Overview
+
+whoosh-compat (github.com/stumpylog/whoosh-compat; local checkout usually at `../whoosh-compat`) replaces the hand-maintained translation layer (`src/documents/search/_translate.py`, `_dates.py`): it parses user queries with a faithful fork of whoosh's real grammar into a typed AST and emits programmatic tantivy queries. Read its README and ARCHITECTURE.md before wiring anything; its DIVERGENCES.md lists intended behavior differences and is the authority on "is this difference a bug".
+
+## Decisions already made (do not re-derive)
+
+- **Queries are user-typed free text.** The advanced search box passes whatever the user types straight to the parser (that is how the issue #13568 queries exist). Do NOT try to infer the supported field surface from frontend code; the frontend only generates a few date filter strings, everything else is typed by users.
+- **The field surface is a policy decision, not `KNOWN_FIELDS`.** Today's `KNOWN_FIELDS` accepts internal ID fields (`tag_id`, `owner_id`, `viewer_id`, other `*_id`) that are undocumented in `docs/usage.md` and were ruled not user-searchable by the maintainer: exclude them from the `FieldRegistry` (they stay as programmatic permission/filter fields in `build_permission_filter`, which never touches user query text). The registry is built from documented syntax in `docs/usage.md` plus the v2-compat aliases (`type`, `path`, `type_id`-style aliases follow their canonical field's fate). Undocumented-but-working fields (`asn`, `page_count`, `num_notes`, `original_filename`, `checksum`) need an explicit maintainer yes/no; since users type freely, silently dropping one breaks any saved view using it, so a drop must be a visible, documented decision.
+- **Analyzer seam:** `FieldSpec.analyzer` binds the live registered tantivy analyzer's `.analyze` (the same Rust analyzer used at index time; language-keyed, so rebuild the registry when `SEARCH_LANGUAGE` changes, on the same trigger as `register_tokenizers`). `pattern_normalizer` is `_tokenizer.ascii_fold`: character-level lowercase+fold only, NEVER stemming.
+- **Diagnostics before emit:** `whoosh_compat.parse()` never raises on bad input. Check `ParseResult.diagnostics` and map to `SearchQueryError`/`InvalidDateQuery` (HTTP 400) BEFORE calling `emit()`; also catch the emitter's `UnsupportedQueryError` into a 400. Never carry forward the legacy raw-string fallback (`except Exception: query_str = raw_query`) into the new path; it masks integration bugs.
+- **Build typed errors from structured diagnostic data, never by parsing `message`.** Each `Diagnostic` carries `kind`, `startchar`/`endchar`, and `field`/`raw_value`. `message` is human-readable text whose wording can change. For a range that fails on one bound, `raw_value` is the bound that actually failed.
+  - `diagnostic.field` is a **`FieldRef`**, not a string: use `str(diagnostic.field)` for the canonical dotted name (`created`, `notes.user`) or `diagnostic.field.name` for the field alone. Note the name is canonical, so an aliased query (`type:`) reports the field it resolves to (`document_type`), and the diagnostic span covers the offending value rather than the field name, so the text the user typed for the field is not recoverable.
+- **The registry has one resolver.** `registry.make_ref(raw)` turns a raw field string into a `FieldRef` or `None` for an unknown field, and `registry.resolve(ref)` returns the spec. There is no `resolve_json()`; a dotted name is interpreted only inside `make_ref`.
+- **`notes` and `custom_fields` are JSON fields** with fixed subpaths (`notes.user`/`notes.note`, `custom_fields.name`/`custom_fields.value`); the registry stays a static, language-keyed singleton, never per-request.
+
+## Mandatory before deleting old code
+
+- Date-grammar parity audit, line by line: every keyword, relative unit, and abbreviation `_dates.py` and `_translate.py` accept today (including the whoosh-era abbreviations kept for old saved views) must have an accepted form in whoosh-compat's dateparse grammar. Silent keyword loss is the saved-view breakage class behind issue #13568.
+- Acceptance corpus compared by matched-document-ID sets, not query strings: the #13568 queries verbatim, real saved-view strings, every date keyword, field aliases, comma lists, date and numeric ranges, wildcards with bracket classes, boosts, JSON subpaths.
+
+## Tests: what goes, what comes
+
+Removed with their modules (do not port their string-level assertions):
+
+- `src/documents/tests/search/test_translate.py`: its subject is deleted; string-translation unit cases are whoosh-compat's own responsibility now. Cases that encode real user-visible behavior get reincarnated as result-level acceptance cases, not string assertions.
+- Date-keyword unit tests tied to `_dates.py` internals: same treatment.
+- `test_query.py` cases asserting `parse_user_query` internals or intermediate query strings: rewritten against the new pipeline, asserting on matched results.
+
+Kept: `test_migration_fulltext_query_field_prefixes.py` (data migration, orthogonal), `test_schema.py`, `test_tokenizer.py`, permission-filter and simple-search tests.
+
+Added:
+
+- A result-level acceptance module (paperless's analogue of whoosh-compat's `test_acceptance_e2e.py`): the corpus above against a real index built from `build_schema()`, asserting document-ID sets. Use `pytest.param(..., id="...")` for every case.
+- Registry unit tests: internal `*_id` names rejected, aliases resolve to canonical fields, JSON subpaths match `docs/usage.md`, construction deterministic per language.
+- One `Multitoken` case nested inside a top-level `OR` (whoosh-compat DIVERGENCES entry on Multitoken.DEFAULT) to prove it does not matter for paperless's data.
+- If acceptance work surfaces a new whoosh-compat divergence, that is a whoosh-compat-repo change (its `differential-triage` skill applies), not a silent paperless workaround.
+
+## Do not mark the JSON fields fast
+
+`notes` and `custom_fields` must stay non-fast for now. Existence checks against a **fast** JSON field currently return inverted results (a document that has the field is reported as not having it), because the underlying call does not check subpath columns by default. The unsupported-configuration error raised for a non-fast JSON field advises marking it fast, which walks directly into that bug. Ignore that advice until the upstream issue is fixed, then re-check.
+
+## Coordination
+
+- whoosh-compat is pre-1.0: pin an exact version or git SHA; upgrades are deliberate, reviewed changes.
+- JSON subpath emission depends on the installed tantivy-py version (fallback until quickwit-oss/tantivy-py#716 ships). The whoosh-compat repo has a `carve-out-retirement` skill; coordinate tantivy pin bumps with it, in a separate PR from the parser migration.
+- Rollout: settings flag defaulting to the legacy path plus shadow-compare logging (log when old and new paths return different ID sets; sample if cost matters) for one release; delete `_translate.py`/`_dates.py` only after the flag defaults to the new path with no material reports.
+
+## Common mistakes
+
+- Inferring the field surface from frontend code (users type queries directly).
+- Copying `KNOWN_FIELDS` into the registry wholesale (resurfaces internal fields).
+- Wiring stemming into `pattern_normalizer`.
+- Calling `emit()` unconditionally, or porting the legacy raw-string fallback.
+- Deleting `_dates.py` without the parity audit.
+- Porting `test_translate.py`'s string assertions instead of writing result-level tests.
@@ -699,7 +699,6 @@ document_fuzzy_match [--ratio] [--processes N]
 | --ratio     | No       | 85.0                | a number between 0 and 100, setting how similar a document must be for it to be reported. Higher numbers mean more similarity. |
 | --processes | No       | 1/4 of system cores | Number of processes to use for matching. Setting 1 disables multiple processes                                                 |
 | --delete    | No       | False               | If provided, one document of a matched pair above the ratio will be deleted.                                                   |
-| --url       | No       | blank               | If an instance URL is provided, the output table will show URLs to each documents instead of the document ID and name.         |

 !!! warning

@@ -948,11 +948,10 @@ for display in the web interface.

    !!! note

-        The **remote OCR parser** (Azure AI) also honors this setting: when
-        no archive is requested (`never`, or `auto` with a born-digital PDF),
-        the remote engine is skipped entirely and locally-extracted text is
-        used instead, avoiding an unnecessary API call and a duplicate text
-        layer.
+        The **remote OCR parser** (Azure AI) always produces a searchable
+        PDF and stores it as the archive copy, regardless of this setting.
+        `ARCHIVE_FILE_GENERATION=never` has no effect when the remote
+        parser handles a document.

 #### [`PAPERLESS_OCR_CLEAN=<mode>`](#PAPERLESS_OCR_CLEAN) {#PAPERLESS_OCR_CLEAN}

@@ -187,11 +187,10 @@ PAPERLESS_ARCHIVE_FILE_GENERATION=auto

 ### Remote OCR parser

-If you use the **remote OCR parser** (Azure AI), `ARCHIVE_FILE_GENERATION` is
-honored the same way as for the local engine: when no archive is requested
-(`never`, or `auto` with a born-digital PDF), the remote engine is skipped
-entirely and locally-extracted text is used instead, avoiding an unnecessary
-API call and a duplicate text layer.
+If you use the **remote OCR parser** (Azure AI), note that it always produces a
+searchable PDF and stores it as the archive copy. `ARCHIVE_FILE_GENERATION=never`
+has no effect for documents handled by the remote parser - the archive is produced
+unconditionally by the remote engine.

 ## Search Index (Whoosh -> Tantivy)

@@ -0,0 +1,455 @@
+# whoosh-compat transition design
+
+Date: 2026-08-07
+Status: approved, pending spec review
+Related skill: `whoosh-compat-transition`
+Related issue: [stumpylog/whoosh-compat#1](https://github.com/stumpylog/whoosh-compat/issues/1)
+
+> ## Read this before implementing: parts of this spec describe an API that has moved
+>
+> whoosh-compat changed after this spec was written, and more changes are
+> already decided. Re-check anything below against the library's own source
+> and `ARCHITECTURE.md` before writing code from it. The library repo is the
+> source of truth; this document is not.
+>
+> **Wrong today, fix on sight:**
+>
+> | This spec says                                                   | Reality now                                                                                                                                                                                                    |
+> | ---------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+> | `FieldRegistry.resolve_json(dotted)` (§ JSON subpath resolution) | Gone. There is one resolver: `registry.make_ref(raw) -> FieldRef \| None` interprets a dotted name, and `registry.resolve(ref) -> FieldSpec \| None` looks it up. A dot is interpreted only inside `make_ref`. |
+> | `InvalidDateQuery(d.field, d.raw_value)`                         | `Diagnostic.field` is now a `FieldRef`, not a string. Use `str(d.field)` for the canonical dotted name, or `d.field.name`. The name is canonical, so an aliased query (`type:`) reports `document_type`.       |
+> | AST leaves carrying a field name string                          | Every field-carrying AST leaf now holds a `FieldRef`.                                                                                                                                                          |
+>
+> **Decided upstream, not yet implemented. Write toward these, they will land before this migration executes:**
+>
+> | Behavior                                                                                                                           | Issue                                                       |
+> | ---------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
+> | An empty or unknown `default_fields` raises at `parse()`; a `field_boosts` key naming an alias resolves, one naming nothing raises | [#20](https://github.com/stumpylog/whoosh-compat/issues/20) |
+> | A naive `basedate` is rejected rather than read in the host machine's timezone                                                     | [#19](https://github.com/stumpylog/whoosh-compat/issues/19) |
+> | A wildcard on a numeric field produces a diagnostic instead of failing at search time, so the error mapping gains a case           | [#17](https://github.com/stumpylog/whoosh-compat/issues/17) |
+> | A bare JSON field name (`notes:foo`) demotes to a text search rather than raising at emit                                          | [#11](https://github.com/stumpylog/whoosh-compat/issues/11) |
+> | Registry construction rejects exists-target cycles, empty names, duplicate aliases within a spec, and dotted canonical names       | [#21](https://github.com/stumpylog/whoosh-compat/issues/21) |
+>
+> **Still undecided, do not guess:** whether `emit()` keeps its `schema`
+> parameter ([#27](https://github.com/stumpylog/whoosh-compat/issues/27)).
+> This spec calls `emit(ast, index=index, schema=schema, registry=...)` in two
+> places. Check the issue before writing either call site.
+>
+> **Trap:** do not add `fast=True` to the `notes` or `custom_fields` JSON
+> specs. Existence checks against a fast JSON field currently return inverted
+> results ([#7](https://github.com/stumpylog/whoosh-compat/issues/7)), and the
+> error raised for a non-fast JSON field advises marking it fast, which walks
+> straight into that bug. The field table below correctly leaves them non-fast.
+>
+> [#28](https://github.com/stumpylog/whoosh-compat/issues/28) tracks all open
+> upstream work ordered by effort.
+
+## Summary
+
+Replace paperless-ngx's hand-maintained query-translation layer
+(`src/documents/search/_translate.py`, `src/documents/search/_dates.py`)
+with [whoosh-compat](https://github.com/stumpylog/whoosh-compat): a typed
+Whoosh-grammar parser that emits programmatically constructed
+`tantivy.Query` objects instead of building an intermediate Tantivy query
+_string_. The integration point is narrow: `parse_user_query()` in
+`src/documents/search/_query.py` is the only function whose implementation
+changes; `_backend.py`, `_tokenizer.py`, simple/title search, CJK handling,
+and permission filtering are all unaffected.
+
+Delivered as a stack of four paperless-ngx PRs plus one prerequisite change
+in whoosh-compat itself (same maintainer, no cross-repo coordination
+overhead), landed with no feature flag and no shadow-compare rollout period
+— safety comes from a result-level acceptance test corpus and a
+date-grammar parity audit instead.
+
+## Architecture
+
+```
+raw_query (user-typed)
+    │
+    ▼
+wc.parse(raw_query, registry=FIELD_REGISTRY, default_fields=DEFAULT_SEARCH_FIELDS,
+         field_boosts=_FIELD_BOOSTS, tz=tz)
+    │
+    ▼
+ParseResult(ast, diagnostics)
+    │
+    ├─ diagnostics non-empty? → map ALL diagnostics to SearchQueryError
+    │  subclass(es) → HTTP 400 (never just the first diagnostic)
+    │
+    ▼
+emit(ast, index=index, schema=schema, registry=FIELD_REGISTRY)
+    │  (raises UnsupportedQueryError → mapped to SearchQueryError → 400,
+    │   for constructs that parse but can't execute against tantivy)
+    ▼
+tantivy.Query
+    │
+    ▼
+existing clause assembly in parse_user_query(): Should(exact) + optional
+fuzzy re-parse of raw_query + optional CJK bigram query, unchanged from today
+    │
+    ▼
+_apply_permission_filter() in _backend.py wraps the result with
+build_permission_filter() — entirely independent of whoosh-compat, unchanged
+```
+
+Permission filtering is explicitly out of scope for this migration:
+`build_permission_filter()` builds its `tantivy.Query` directly against
+`owner_id`/`viewer_id`/`viewer_group_id`, never through the parser or
+registry, and those fields are exactly the internal `*_id` fields excluded
+from the `FieldRegistry` (see "Field surface" below). Nothing in this
+migration's diff touches it.
+
+## PR stack
+
+Each PR is independently buildable, reviewable, and CI-able; later PRs
+rebase on earlier ones. No PR depends on whoosh-compat behavior it hasn't
+already proven correct in isolation.
+
+1. **Refactor `_schema.py` to a shared field-definition table.** Pure
+   refactor — `build_schema()`'s output is byte-identical before and after.
+   `test_schema.py` (existing) proves it.
+2. **Pin whoosh-compat as a real dependency; build `FieldRegistry`.** New
+   `_registry.py` built from the same table PR 1 introduced. Registry unit
+   tests only — no wiring into search yet.
+3. **Date-grammar parity audit.** A transitional, executable differential
+   test using the still-present `_dates.py`/`_translate.py` as the oracle.
+   Any gap found is fixed in whoosh-compat directly before this PR closes.
+   A whoosh-compat PyPI release is expected around this point (see
+   "Dependency pinning").
+4. **Wire it in; delete the old path.** Rewrite `parse_user_query()`,
+   diagnostics→exception mapping, add the result-level acceptance corpus,
+   expand `test_api_search.py`, delete `_translate.py`/`_dates.py`/
+   `test_translate.py` and the internals-testing classes in `test_query.py`,
+   update `docs/usage.md` and changelog.
+
+**Prerequisite, whoosh-compat repo** (tracked as
+[stumpylog/whoosh-compat#1](https://github.com/stumpylog/whoosh-compat/issues/1),
+lands before PR 4 starts its diagnostics-mapping work): add `field: str |
+None` and `raw_value: str | None` to `Diagnostic`, threaded through at its
+three construction sites (`dateparse.py`'s `_error()`, `default.py`'s
+`term_query()` and `_coerce_range_bound()`), so paperless can build typed
+exceptions without parsing whoosh-compat's human-readable `message` text.
+
+## Field surface
+
+The `FieldRegistry` covers only query-syntax-addressable fields — a subset
+of the full Tantivy schema. Internal-only schema fields with no query-syntax
+meaning of their own (`title_sort`/`correspondent_sort`/`type_sort` shadow
+sort fields, `bigram_*` CJK fields, `simple_title`/`simple_content`,
+`autocomplete_word`, `notes_text`) stay hardcoded `sb.add_*` calls in
+`_schema.py`, untouched by the shared table.
+
+**Decision: keep and document all five currently-undocumented-but-working
+fields** (`asn`, `page_count`, `num_notes`, `original_filename`,
+`checksum`) rather than dropping them — least risk of silently breaking an
+existing saved view. `docs/usage.md`'s advanced-search section gets these
+added with examples, as part of PR 4.
+
+**Decision: `archive_checksum` stays out of scope.** Unlike `checksum`, it
+isn't indexed in the Tantivy schema at all today (confirmed: `_schema.py`
+only adds `checksum`; `_build_tantivy_doc` only calls
+`doc.add_text("checksum", document.checksum)`). Making it searchable is a
+schema-level change (new indexed field, new document population code), not
+a parser-migration concern — left as a separate follow-up.
+
+**Decision: internal `*_id` fields (`tag_id`, `correspondent_id`,
+`document_type_id`, `storage_path_id`, `owner_id`, `viewer_id`,
+`viewer_group_id`) are excluded from the `FieldRegistry` entirely.** They
+remain Tantivy-schema-only, used exclusively by `build_permission_filter()`.
+Because whoosh-compat folds any unrecognized `field:` prefix into literal
+text (Whoosh-parity leniency, confirmed in `FieldsPlugin.do_fieldnames` —
+not an error), a saved view typed as `tag_id:5` won't 400: it silently
+becomes a text search for the literal string `tag_id:5`, most likely
+returning zero results. This is a real behavior change and gets a
+**changelog callout**, not just a docs update, since a docs addition alone
+wouldn't surface it to someone skimming release notes.
+
+## Shared field-definition table (`_fields.py`)
+
+```python
+from whoosh_compat import FieldKind  # reused directly — no parallel enum
+
+@dataclass(frozen=True, slots=True)
+class PublicField:
+    name: str
+    kind: FieldKind
+    aliases: tuple[str, ...] = ()
+    comma_values: bool = False
+    date_only: bool = False
+    fast: bool = False
+    subpaths: tuple[str, ...] = ()   # JSON kind only
+
+PUBLIC_FIELDS = (
+    PublicField("title", FieldKind.TEXT),
+    PublicField("content", FieldKind.TEXT),
+    PublicField("correspondent", FieldKind.TEXT),
+    PublicField("document_type", FieldKind.TEXT, aliases=("type",)),
+    PublicField("storage_path", FieldKind.TEXT, aliases=("path",)),
+    PublicField("original_filename", FieldKind.TEXT),
+    PublicField("tag", FieldKind.TEXT, comma_values=True),
+    PublicField("checksum", FieldKind.KEYWORD),
+    PublicField("asn", FieldKind.U64, fast=True),
+    PublicField("page_count", FieldKind.U64, fast=True),
+    PublicField("num_notes", FieldKind.U64, fast=True),
+    PublicField("created", FieldKind.DATE, date_only=True, fast=True),
+    PublicField("modified", FieldKind.DATETIME, fast=True),
+    PublicField("added", FieldKind.DATETIME, fast=True),
+    PublicField("notes", FieldKind.JSON, subpaths=("user", "note")),
+    PublicField("custom_fields", FieldKind.JSON, subpaths=("name", "value")),
+)
+```
+
+`build_schema()` derives its `sb.add_*` call and tokenizer from `kind`
+(TEXT/KEYWORD → `add_text_field` with `paperless_text`/`raw` tokenizer
+respectively; U64 → `add_unsigned_field`; DATE/DATETIME → `add_date_field`;
+JSON → `add_json_field`). The `notes_text` snippet-companion field stays a
+separate hardcoded line right after the `notes` entry — schema-only
+plumbing with no query-syntax meaning.
+
+`_registry.py` maps each `PublicField` to a `whoosh_compat.FieldSpec`,
+kept as one flat dataclass (no kind-specific subclassing) to mirror
+whoosh-compat's own `FieldSpec` design, which validates kind-conditional
+attributes (e.g. JSON requires non-empty `subpaths`) at
+`FieldRegistry.__init__` rather than in the type system.
+
+Footnote for whoever writes `_registry.py`: `FieldRegistry.__init__` forces
+`date_only=True` on _any_ `FieldKind.DATE` spec regardless of what's
+passed, unconditionally — `PublicField.date_only` isn't an independent
+knob for DATE fields the way it might look; it only matters in the sense
+that `created` sets it explicitly for clarity, while `modified`/`added`
+use `FieldKind.DATETIME` instead of relying on that override.
+
+**`subpaths` stays `tuple[str, ...]`, not a nested structure.** Confirmed
+against whoosh-compat's own `FieldRegistry.resolve_json()`: it splits a
+dotted query term on the _first_ dot only and matches the remainder as an
+exact string against `spec.subpaths` — even the docstring's own
+`"metadata.author.name"` example is a single opaque string in the tuple,
+not a recursive tree. A tuple of strings is exactly as expressive as the
+library it feeds; inventing richer structure in `PublicField` now would
+just get flattened back to strings at the registry-construction boundary.
+Real recursive nesting, if ever needed, is new whoosh-compat capability
+first.
+
+**JSON document population stays separate from `subpaths`.** `subpaths` is
+query-side only — it declares which dotted names are legal to type and
+which JSON keys the emitter should address. It says nothing about how
+`_backend.py::_build_tantivy_doc` builds the JSON documents at index-write
+time, and that logic isn't uniform attribute access (`note.user.username`
+needs a null guard and isn't `note.user`; `cfi.value_for_search` is a
+property, not a literal `value` attribute), so a generic
+`getattr(obj, subpath_name)` scheme would silently do the wrong thing for
+both. That code stays hand-written, unchanged by this migration. Mitigation
+instead: a coupling test (PR 2, alongside the registry unit tests) asserting
+the literal JSON keys used in `_build_tantivy_doc`'s `doc.add_json(...)`
+calls match `PUBLIC_FIELDS`' `notes`/`custom_fields` `subpaths` exactly, so
+drift between the two is caught rather than silently becoming an
+unqueryable (or silently unindexed) field.
+
+**JSON subpath queries (`notes.*`, `custom_fields.*`) route through
+`index.parse_query()`, not programmatic construction, given paperless's
+pinned tantivy version.** Installed `tantivy-py`'s `Query.term_query`
+cannot resolve a JSON subpath by exact field name — it raises as if the
+field didn't exist. Until
+[tantivy-py#716](https://github.com/quickwit-oss/tantivy-py/pull/716) lands
+and ships, whoosh-compat's `TantivyEmitter._json_paths_supported()` feature-
+detects this per process and falls back to a strictly escaped, single-leaf
+`index.parse_query()` call for just that one leaf (whoosh-compat's README/
+ARCHITECTURE.md call this out as "the JSON subpath carve-out"). Paperless
+pins `tantivy~=0.26.0`, squarely inside the affected range (whoosh-compat's
+`tantivy` extra only requires `tantivy>=0.24`, so nothing prevents this
+combination). Nothing needs to change in this design because of it — the
+carve-out is self-retiring on whoosh-compat's side once tantivy-py catches
+up — but the acceptance corpus's `notes.user:`/`custom_fields.name:` cases
+(PR 4) are exercising that fallback escaping path specifically, not the
+programmatic path every other field goes through, and that's worth knowing
+if one of those cases ever behaves oddly around quoting/escaping.
+
+**Analyzer wiring**: `FieldSpec.analyzer` reuses the same `tantivy
+.TextAnalyzer` objects `_tokenizer.py` already builds (`_paperless_text
+(language)`, etc.) — standalone objects not dependent on index
+registration, so `_registry.py` calls the same builder functions and binds
+`.analyze` directly; `checksum` (KEYWORD, `raw` tokenizer) gets an identity
+analyzer (`lambda t: [t]`). `pattern_normalizer` for every field is
+`_tokenizer.ascii_fold` (character-fold only, never stemming) per the
+skill's explicit instruction. The whole `FieldRegistry` is built once,
+cached keyed by `settings.SEARCH_LANGUAGE`, rebuilt on the same trigger
+`register_tokenizers()` already uses.
+
+## Error handling
+
+```python
+class SearchQueryError(ValueError): ...          # unchanged, base
+
+class InvalidDateQuery(SearchQueryError):         # unchanged
+    def __init__(self, field, value): ...
+
+class InvalidNumberQuery(SearchQueryError):       # new
+    def __init__(self, field: str | None, value: str | None) -> None:
+        self.field = field
+        self.value = value
+        super().__init__(f"Invalid numeric value {value!r} for field {field!r}.")
+
+class MultipleSearchQueryErrors(SearchQueryError):  # new
+    """Aggregates every user-fixable error from one parse, not just the first."""
+    def __init__(self, errors: Sequence[SearchQueryError]) -> None:
+        self.errors = tuple(errors)
+        super().__init__("; ".join(str(e) for e in self.errors))
+```
+
+```python
+def parse_user_query(index, raw_query, tz):
+    registry = get_field_registry(settings.SEARCH_LANGUAGE)
+    result = wc.parse(
+        raw_query, registry=registry, default_fields=DEFAULT_SEARCH_FIELDS,
+        field_boosts=_FIELD_BOOSTS, tz=tz,
+    )
+    if result.diagnostics:
+        raise _diagnostics_to_error(result.diagnostics)   # ALL diagnostics, not [0]
+
+    try:
+        exact = tantivy_emit(result.ast, index=index, schema=index.schema, registry=registry)
+    except UnsupportedQueryError as e:
+        raise SearchQueryError(str(e)) from e
+
+    # CJK: unchanged — already re-parses raw_query directly via index.parse_query,
+    # never went through translate_query, so nothing here changes.
+    cjk_query = _build_cjk_query(index, raw_query, _CJK_ALL_FIELDS) if _has_cjk(raw_query) else None
+
+    clauses = [(tantivy.Occur.Should, exact)]
+    threshold = settings.ADVANCED_FUZZY_SEARCH_THRESHOLD
+    if threshold is not None:
+        # Fuzzy re-parses raw_query (not the AST) — no clean AST-level fuzzy
+        # equivalent exists; fuzzy matching was always an approximate,
+        # secondary clause, so this divergence from the exact-match path is
+        # acceptable.
+        fuzzy = index.parse_query(raw_query, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS,
+                                   fuzzy_fields={f: (True, 1, True) for f in DEFAULT_SEARCH_FIELDS})
+        clauses.append((tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)))
+    if cjk_query is not None:
+        clauses.append((tantivy.Occur.Should, cjk_query))
+
+    return exact if len(clauses) == 1 else tantivy.Query.boolean_query(clauses)
+
+
+def _diagnostics_to_error(diagnostics: tuple[Diagnostic, ...]) -> SearchQueryError:
+    errors = [_single_diagnostic_to_error(d) for d in diagnostics]
+    return errors[0] if len(errors) == 1 else MultipleSearchQueryErrors(errors)
+
+
+def _single_diagnostic_to_error(d: Diagnostic) -> SearchQueryError:
+    if d.kind is DiagnosticKind.BAD_DATE:
+        return InvalidDateQuery(d.field, d.raw_value)
+    if d.kind is DiagnosticKind.BAD_NUMBER:
+        return InvalidNumberQuery(d.field, d.raw_value)
+    return SearchQueryError(d.message)
+```
+
+No `except Exception: query_str = raw_query` fallback — per the skill, that
+legacy defensive branch is explicitly not carried forward. A bug in the new
+path must surface as a real error, not silently degrade to stale behavior.
+
+`views.py`'s existing `except SearchQueryError as e: raise
+ValidationError({"query": [str(e)]}) from e` handler gets one added branch
+to surface every aggregated message instead of just one:
+
+```python
+except SearchQueryError as e:
+    messages = [str(sub) for sub in e.errors] if isinstance(e, MultipleSearchQueryErrors) else [str(e)]
+    raise ValidationError({"query": messages}) from e
+```
+
+`d.field`/`d.raw_value` depend on the whoosh-compat prerequisite change
+(issue #1) landing first; until then (or if `field`/`raw_value` are `None`
+for a given diagnostic kind not yet covered), `_single_diagnostic_to_error`
+falls back to `SearchQueryError(d.message)`.
+
+Deferred, explicitly out of scope for this PR stack: any frontend use of
+`startchar`/`endchar` (already present on `Diagnostic` today) to highlight
+the offending span in the search box. Backend-only for now, per explicit
+decision.
+
+## Testing
+
+Existing test inventory (`src/documents/tests/search/` and
+`test_api_search.py`):
+
+| File                                                                                                                                                                                          | Fate                                                                                                                                                                       |
+| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `test_translate.py`                                                                                                                                                                           | Deleted (PR 4) — subject deleted                                                                                                                                           |
+| `test_query.py`: `TestCreatedDateField`, `TestDateTimeFields`, `TestWhooshQueryRewriting`, `TestYearRangeRewriting`, `TestNonDateFieldsNotRewritten`, `TestPassthrough`, `TestNormalizeQuery` | Deleted (PR 4) — test `_translate.py`/`_dates.py` internals or intermediate query strings                                                                                  |
+| `test_query.py`: `TestParseUserQuery`                                                                                                                                                         | Reviewed at plan time; result-level assertions folded into the new acceptance module, internals-only assertions dropped                                                    |
+| `test_query.py`: `TestParseSimpleTextHighlightQuery`, `TestPermissionFilter`                                                                                                                  | Unchanged — never touched `translate_query`                                                                                                                                |
+| `test_schema.py`, `test_tokenizer.py`, `test_backend.py`, `test_lock_backoff.py`, `test_migration_fulltext_query_field_prefixes.py`                                                           | Unchanged                                                                                                                                                                  |
+| `test_api_search.py` (`TestDocumentSearchApi`, 43 tests)                                                                                                                                      | **Stays green across every PR in the stack** (hard gate, not just PR 4) — full HTTP+DB+index integration coverage catches wiring mistakes none of the narrower tests would |
+
+New tests per PR:
+
+- **PR 2**: `test_registry.py` — internal `*_id` names rejected; `type`/
+  `path` aliases resolve to canonical fields; JSON subpaths match
+  `docs/usage.md`; registry construction deterministic per language; the
+  `notes`/`custom_fields` dict-key coupling test described above.
+- **PR 3**: `test_date_grammar_parity.py` — transitional, parametrized over
+  every keyword/unit `_dates.py`/`_translate.py` accept today
+  (`_DATE_KEYWORDS`, all of `_UNIT_ALIASES`'s Whoosh-era abbreviations —
+  `yrs`/`mos`/`wks`/`hrs`/`mins`/`secs` etc. — digit-precision forms, ISO
+  dash forms, `now-7d`/`now+1h`/`now-30m` compact offsets, open/reversed
+  ranges). Each case parses through
+  `wc.parse()` against a DATE-kind `FieldRegistry` and asserts no
+  diagnostics _and_ bounds matching what `_dates.py`/`_translate.py`
+  compute today, using the still-present legacy code as the oracle.
+  Deleted again in PR 4 along with that oracle, superseded by the
+  permanent acceptance corpus. This audit is scoped to _parity_ only —
+  whoosh-compat's date grammar is a strict superset of what `_dates.py`
+  accepts today (e.g. `tomorrow`, `now`, `midnight`, `noon`, weekday names
+  like `next monday`), so the migration also grants new date vocabulary for
+  free. That's a nice side effect, not something this PR needs to test or
+  document beyond noting it in the changelog alongside the other behavior
+  changes.
+- **PR 4**:
+  - Result-level acceptance module (paperless's analogue of whoosh-compat's
+    `test_acceptance_e2e.py`): a real index built via `build_schema()`, the
+    issue #13568 queries verbatim, real saved-view strings, every date
+    keyword/unit, field aliases, comma lists, numeric/date ranges,
+    bracket-class wildcards, boosts, JSON subpaths — asserted by matched
+    document-ID set, `pytest.param(..., id=...)` per case. Plus a
+    multi-diagnostic case (two bad fields → `MultipleSearchQueryErrors`
+    with both messages present) and one `Multitoken` case nested inside a
+    top-level `OR` (proves DIVERGENCES entry 15 doesn't matter for
+    paperless's data, per the skill).
+  - `test_api_search.py` expanded: a multi-bad-field query (e.g.
+    `created:notadate AND asn:notanumber`) asserting the 400 response's
+    `query` list contains both messages; end-to-end searches on the five
+    newly-documented fields (`asn:`, `page_count:`, `num_notes:`,
+    `original_filename:`, `checksum:`) returning the right documents
+    through the real index.
+
+## Dependency pinning
+
+Stays `path = "../whoosh-compat"` in `[tool.uv.sources]` through the whole
+PR stack — both repos are being actively co-developed. The final swap
+happens at PR 4:
+
+- **Primary plan**: whoosh-compat is released to PyPI around PR 3 (per
+  your stated intent), assuming the parity audit and issue #1 don't turn
+  up anything else needing a second round. PR 4 switches to a pinned PyPI
+  version (`whoosh-compat[tantivy]==X.Y.Z` in `dependencies`, the
+  `[tool.uv.sources]` override removed entirely).
+- **Fallback**: if the PyPI release slips past PR 4's start, pin an exact
+  git commit SHA instead (`whoosh-compat[tantivy] @ git+https://github.com/
+stumpylog/whoosh-compat@<sha>`), per the skill's "pre-1.0: pin an exact
+  version or git SHA, upgrades are deliberate" guidance.
+
+The `TODO` comment already sitting in `pyproject.toml` (from the earlier
+smoke-test setup) gets updated to reflect this — "release, else pinned SHA"
+— rather than committing hard to one path before it's known which applies.
+
+## Explicitly out of scope
+
+- `archive_checksum` indexing/search (separate schema-level follow-up).
+- Frontend consumption of `Diagnostic.startchar`/`endchar` for in-box error
+  highlighting (backend-only for this PR stack).
+- A feature flag or shadow-compare rollout period — explicitly decided
+  against; safety comes from the acceptance corpus and parity audit instead.
+- Any change to `build_permission_filter()`/`_apply_permission_filter()` —
+  confirmed untouched by this migration.
@@ -576,9 +576,7 @@ The following workflow action types are available:
 - Tags, correspondent, document type and storage path
 - Document owner
 - View and / or edit permissions to users or groups
- Custom fields, optionally with a value. If no value is set, the field is only added to the
-  document and any value it may already have is left untouched. If a value is set, it will
-  overwrite an existing value of that field on the document.
+- Custom fields. Note that no value for the field will be set

 ##### Removal {#workflow-action-removal}

@@ -111,7 +111,7 @@
                    routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
                    [disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
                    popoverClass="popover-slim">
-                    <i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
+                    <i-bs class="me-2" name="funnel"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
                        @if (showSidebarCounts && !slimSidebarEnabled) {
                          <span class="badge bg-info text-dark ms-2 d-inline">{{ savedViewService.getDocumentCount(view) }}</span>
                        }
@@ -52,10 +52,10 @@ describe('CustomFieldsValuesComponent', () => {
  })

  it('should set selectedFields and map values correctly', () => {
-    component.value = { 1: 'value1', 3: 0, 4: false }
-    component.selectedFields = [1, 2, 3, 4]
-    expect(component.selectedFields).toEqual([1, 2, 3, 4])
-    expect(component.value).toEqual({ 1: 'value1', 2: null, 3: 0, 4: false })
+    component.value = { 1: 'value1' }
+    component.selectedFields = [1, 2]
+    expect(component.selectedFields).toEqual([1, 2])
+    expect(component.value).toEqual({ 1: 'value1', 2: null })
  })

  it('should return the correct custom field by id', () => {
@@ -77,7 +77,7 @@ export class CustomFieldsValuesComponent extends AbstractInputComponent<Object>
    this._selectedFields = newFields
    // map the selected fields to an object with field_id as key and value as value
    this.value = newFields.reduce((acc, fieldId) => {
-      acc[fieldId] = this.value?.[fieldId] ?? null
+      acc[fieldId] = this.value?.[fieldId] || null
      return acc
    }, {})
    this.onChange(this.value)
@@ -36,16 +36,7 @@
          (focus)="clearLastSearchTerm()"
          (clear)="clearLastSearchTerm()"
          (blur)="onBlur()">
-          <ng-template ng-label-tmp let-item="item">
-            @if (iconField && item[iconField]) {
-              <i-bs class="me-2" [name]="item[iconField]"></i-bs>
-            }
-            <span [title]="item[bindLabel]">{{item[bindLabel]}}</span>
-          </ng-template>
          <ng-template ng-option-tmp let-item="item">
-              @if (iconField && item[iconField]) {
-                <i-bs class="me-2" [name]="item[iconField]"></i-bs>
-              }
              <span [title]="item[bindLabel]">{{item[bindLabel]}}</span>
          </ng-template>
        </ng-select>
@@ -35,7 +35,7 @@ import { AbstractInputComponent } from '../abstract-input'
    NgxBootstrapIconsModule,
  ],
 })
-export class SelectComponent extends AbstractInputComponent<number | string> {
+export class SelectComponent extends AbstractInputComponent<number> {
  constructor() {
    super()
    this.addItemRef = this.addItem.bind(this)
@@ -100,9 +100,6 @@ export class SelectComponent extends AbstractInputComponent<number | string> {
  @Input()
  bindLabel: string = 'name'

-  @Input()
-  iconField: string
-
  public searchFn = (term: string, item: any): boolean =>
    matchesSearchText(item?.[this.bindLabel], term)

@@ -1,7 +1,6 @@
 <pngx-widget-frame
  *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Document }"
  [title]="savedView.name"
-  [titleIcon]="savedView.icon || 'funnel'"
  [loading]="false"
  [draggable]="savedView"
  >
@@ -8,12 +8,7 @@
              <i-bs name="grip-vertical"></i-bs>
            </div>
          }
-          <h6 class="card-title mb-0">
-            @if (titleIcon()) {
-              <i-bs class="me-2" [name]="titleIcon()"></i-bs>
-            }
-            {{title()}}
-          </h6>
+          <h6 class="card-title mb-0">{{title()}}</h6>
          <ng-content select="[title-badge]"></ng-content>
          @if (badge() !== null && badge() !== undefined) {
            <span class="badge bg-info text-dark ms-2">{{badge()}}</span>
@@ -16,8 +16,6 @@ export class WidgetFrameComponent implements AfterViewInit {

  title = input<string>()

-  titleIcon = input<string>()
-
  draggable = input<any>()

  cardless = input(false)
@@ -97,9 +97,7 @@
    <div class="dropdown-menu shadow dropdown-menu-right" ngbDropdownMenu>
      @if (!list.activeSavedViewId) {
        @for (view of savedViewService.allViews; track view) {
-          <button ngbDropdownItem (click)="loadViewConfig(view.id)">
-            <i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs>{{view.name}}
-          </button>
+          <button ngbDropdownItem (click)="loadViewConfig(view.id)">{{view.name}}</button>
        }
        @if (savedViewService.allViews.length > 0) {
          <div class="dropdown-divider"></div>
@@ -457,7 +457,6 @@ export class DocumentListComponent
      modal.componentInstance.buttonsEnabled.set(false)
      let savedView: SavedView = {
        name: formValue.name,
-        icon: formValue.icon,
        filter_rules: this.list.filterRules,
        sort_reverse: this.list.sortReverse,
        sort_field: this.list.sortField,
@@ -6,14 +6,6 @@
  </div>
  <div class="modal-body">
    <pngx-input-text i18n-title title="Name" formControlName="name" [error]="error()?.name" autocomplete="off"></pngx-input-text>
-    <pngx-input-select
-      i18n-title
-      title="Icon"
-      formControlName="icon"
-      [items]="savedViewIcons"
-      iconField="icon"
-      [error]="error()?.icon">
-    </pngx-input-select>
    <pngx-input-check i18n-title title="Show in sidebar" formControlName="showInSideBar"></pngx-input-check>
    <pngx-input-check i18n-title title="Show on dashboard" formControlName="showOnDashboard"></pngx-input-check>
    <pngx-permissions-form accordion="true" formControlName="permissions_form"></pngx-permissions-form>
@@ -9,7 +9,6 @@ import { CheckComponent } from '../../common/input/check/check.component'
 import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
 import { PermissionsGroupComponent } from '../../common/input/permissions/permissions-group/permissions-group.component'
 import { PermissionsUserComponent } from '../../common/input/permissions/permissions-user/permissions-user.component'
-import { SelectComponent } from '../../common/input/select/select.component'
 import { TextComponent } from '../../common/input/text/text.component'
 import { SaveViewConfigDialogComponent } from './save-view-config-dialog.component'

@@ -41,7 +40,6 @@ describe('SaveViewConfigDialogComponent', () => {
        ReactiveFormsModule,
        SaveViewConfigDialogComponent,
        TextComponent,
-        SelectComponent,
        CheckComponent,
        PermissionsFormComponent,
        PermissionsUserComponent,
@@ -65,7 +63,6 @@ describe('SaveViewConfigDialogComponent', () => {
    expect(component.defaultName()).toEqual(name)
    expect(result).toEqual({
      name,
-      icon: 'funnel',
      showInSideBar: false,
      showOnDashboard: false,
    })
@@ -97,7 +94,6 @@ describe('SaveViewConfigDialogComponent', () => {
    component.save()
    expect(result).toEqual({
      name,
-      icon: 'funnel',
      showInSideBar: true,
      showOnDashboard: true,
    })
@@ -117,7 +113,6 @@ describe('SaveViewConfigDialogComponent', () => {
    component.save()
    expect(result).toEqual({
      name: '',
-      icon: 'funnel',
      showInSideBar: false,
      showOnDashboard: false,
      permissions_form: permissions,
@@ -13,14 +13,9 @@ import {
  ReactiveFormsModule,
 } from '@angular/forms'
 import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
-import {
-  DEFAULT_SAVED_VIEW_ICON,
-  SAVED_VIEW_ICONS,
-} from 'src/app/data/saved-view-icons'
 import { User } from 'src/app/data/user'
 import { CheckComponent } from '../../common/input/check/check.component'
 import { PermissionsFormComponent } from '../../common/input/permissions/permissions-form/permissions-form.component'
-import { SelectComponent } from '../../common/input/select/select.component'
 import { TextComponent } from '../../common/input/text/text.component'

@Component({
@@ -29,7 +24,6 @@ import { TextComponent } from '../../common/input/text/text.component'
  styleUrls: ['./save-view-config-dialog.component.scss'],
  imports: [
    CheckComponent,
-    SelectComponent,
    TextComponent,
    PermissionsFormComponent,
    FormsModule,
@@ -47,7 +41,6 @@ export class SaveViewConfigDialogComponent implements OnInit {
  public saveClicked = new EventEmitter()

  users: User[]
-  readonly savedViewIcons = SAVED_VIEW_ICONS

  setDefaultName(value: string) {
    this.defaultName.set(value)
@@ -56,7 +49,6 @@ export class SaveViewConfigDialogComponent implements OnInit {

  saveViewConfigForm = new FormGroup({
    name: new FormControl(''),
-    icon: new FormControl(DEFAULT_SAVED_VIEW_ICON),
    showInSideBar: new FormControl(false),
    showOnDashboard: new FormControl(false),
    permissions_form: new FormControl(null),
@@ -73,7 +65,6 @@ export class SaveViewConfigDialogComponent implements OnInit {
    const formValue = this.saveViewConfigForm.value
    const saveViewConfig = {
      name: formValue.name,
-      icon: formValue.icon,
      showInSideBar: formValue.showInSideBar,
      showOnDashboard: formValue.showOnDashboard,
    }
@@ -7,24 +7,15 @@
 </pngx-page-header>
 <form [formGroup]="savedViewsForm" (ngSubmit)="save()">
  <ul class="list-group mb-3" formGroupName="savedViews">
-    @for (view of pagedSavedViews(); track view) {
+    @for (view of savedViews(); track view) {
      <li class="list-group-item py-3">
      <div [formGroupName]="view.id">
        <div class="row">
-          <div class="col-md">
+          <div class="col">
            <pngx-input-text title="Name" formControlName="name"></pngx-input-text>
          </div>
-          <div class="col-md">
-            <pngx-input-select
-            i18n-title
-            title="Icon"
-            formControlName="icon"
-            [items]="savedViewIcons"
-            iconField="icon">
-            </pngx-input-select>
-          </div>
          @if (canSaveSettings) {
-            <div class="col-md">
+            <div class="col">
              <div class="form-check form-switch mt-3">
                <input type="checkbox" class="form-check-input" id="show_on_dashboard_{{view.id}}" formControlName="show_on_dashboard">
                <label class="form-check-label" for="show_on_dashboard_{{view.id}}" i18n>Show on dashboard</label>
@@ -90,11 +81,6 @@
    }
  </ul>

-  <div class="d-flex align-items-center mb-3">
-    <button type="button" (click)="reset()" class="btn btn-outline-secondary mb-2" [disabled]="(isDirty$ | async) === false" i18n>Cancel</button>
-    <button type="submit" class="btn btn-primary ms-2 mb-2" [disabled]="(isDirty$ | async) === false" i18n>Save</button>
-    @if (savedViews()?.length > pageSize) {
-      <ngb-pagination class="ms-auto" [pageSize]="pageSize" [collectionSize]="savedViews().length" [page]="page()" [maxSize]="5" (pageChange)="page.set($event)" size="sm" aria-label="Pagination"></ngb-pagination>
-    }
-  </div>
+  <button type="button" (click)="reset()" class="btn btn-outline-secondary mb-2" [disabled]="(isDirty$ | async) === false" i18n>Cancel</button>
+  <button type="submit" class="btn btn-primary ms-2 mb-2" [disabled]="(isDirty$ | async) === false" i18n>Save</button>
 </form>
@@ -4,7 +4,6 @@ import { provideHttpClientTesting } from '@angular/common/http/testing'
 import { signal } from '@angular/core'
 import { ComponentFixture, TestBed } from '@angular/core/testing'
 import { FormsModule, ReactiveFormsModule } from '@angular/forms'
-import { By } from '@angular/platform-browser'
 import { NgbModal, NgbModule } from '@ng-bootstrap/ng-bootstrap'
 import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
 import { Subject, of, throwError } from 'rxjs'
@@ -26,20 +25,8 @@ import { PageHeaderComponent } from '../../common/page-header/page-header.compon
 import { SavedViewsComponent } from './saved-views.component'

 const savedViews = [
-  {
-    id: 1,
-    name: 'view1',
-    icon: 'archive',
-    show_in_sidebar: true,
-    show_on_dashboard: true,
-  },
-  {
-    id: 2,
-    name: 'view2',
-    icon: 'funnel',
-    show_in_sidebar: false,
-    show_on_dashboard: false,
-  },
+  { id: 1, name: 'view1', show_in_sidebar: true, show_on_dashboard: true },
+  { id: 2, name: 'view2', show_in_sidebar: false, show_on_dashboard: false },
 ]

 describe('SavedViewsComponent', () => {
@@ -170,24 +157,6 @@ describe('SavedViewsComponent', () => {
    expect(patchBody.show_in_sidebar).toBeUndefined()
  })

-  it('should persist a changed icon', () => {
-    const patchSpy = jest.spyOn(savedViewService, 'patchMany')
-    const view = savedViews[0]
-    const iconControl = component.savedViewsForm
-      .get('savedViews')
-      .get(view.id.toString())
-      .get('icon')
-
-    iconControl.setValue('bell')
-    iconControl.markAsDirty()
-    component.save()
-
-    expect(patchSpy.mock.calls[0][0][0]).toMatchObject({
-      id: view.id,
-      icon: 'bell',
-    })
-  })
-
  it('should persist visibility changes to user settings', () => {
    const patchSpy = jest.spyOn(savedViewService, 'patchMany')
    const updateVisibilitySpy = jest
@@ -253,44 +222,6 @@ describe('SavedViewsComponent', () => {
    ).toEqual(view.show_on_dashboard)
  })

-  it('should page saved views, clamp the page if views are removed', () => {
-    const manyViews = Array.from({ length: 30 }, (_, i) => ({
-      id: i + 1,
-      name: `view${i + 1}`,
-    })) as SavedView[]
-    const listSpy = jest.spyOn(savedViewService, 'list').mockReturnValue(
-      of({
-        all: manyViews.map((v) => v.id),
-        count: manyViews.length,
-        results: manyViews.concat([]),
-      })
-    )
-    component.ngOnInit()
-    fixture.detectChanges()
-    expect(listSpy).toHaveBeenCalledWith(1, 100000, null, false, {
-      full_perms: true,
-    })
-    expect(component.pagedSavedViews()).toHaveLength(25)
-    expect(fixture.debugElement.query(By.css('ngb-pagination'))).not.toBeNull()
-    // all views have controls, not just the current page
-    expect(
-      Object.keys(component.savedViewsForm.get('savedViews').value)
-    ).toHaveLength(30)
-
-    component.page.set(2)
-    expect(component.pagedSavedViews()).toHaveLength(5)
-
-    listSpy.mockReturnValue(
-      of({
-        all: manyViews.slice(0, 25).map((v) => v.id),
-        count: 25,
-        results: manyViews.slice(0, 25),
-      })
-    )
-    component.ngOnInit()
-    expect(component.page()).toEqual(1)
-  })
-
  it('should support editing permissions', () => {
    const confirmClicked = new Subject<any>()
    const modalRef = {
@@ -1,29 +1,18 @@
 import { AsyncPipe } from '@angular/common'
-import {
-  Component,
-  OnDestroy,
-  OnInit,
-  computed,
-  inject,
-  signal,
-} from '@angular/core'
+import { Component, OnDestroy, OnInit, inject, signal } from '@angular/core'
 import {
  FormControl,
  FormGroup,
  FormsModule,
  ReactiveFormsModule,
 } from '@angular/forms'
-import { NgbModal, NgbPaginationModule } from '@ng-bootstrap/ng-bootstrap'
+import { NgbModal } from '@ng-bootstrap/ng-bootstrap'
 import { dirtyCheck } from '@ngneat/dirty-check-forms'
 import { NgxBootstrapIconsModule } from 'ngx-bootstrap-icons'
 import { BehaviorSubject, Observable, of, switchMap, takeUntil } from 'rxjs'
 import { PermissionsDialogComponent } from 'src/app/components/common/permissions-dialog/permissions-dialog.component'
 import { DisplayMode } from 'src/app/data/document'
 import { SavedView } from 'src/app/data/saved-view'
-import {
-  DEFAULT_SAVED_VIEW_ICON,
-  SAVED_VIEW_ICONS,
-} from 'src/app/data/saved-view-icons'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import {
  PermissionAction,
@@ -36,7 +25,6 @@ import { ToastService } from 'src/app/services/toast.service'
 import { ConfirmButtonComponent } from '../../common/confirm-button/confirm-button.component'
 import { DragDropSelectComponent } from '../../common/input/drag-drop-select/drag-drop-select.component'
 import { NumberComponent } from '../../common/input/number/number.component'
-import { SelectComponent } from '../../common/input/select/select.component'
 import { TextComponent } from '../../common/input/text/text.component'
 import { PageHeaderComponent } from '../../common/page-header/page-header.component'
 import { LoadingComponentWithPermissions } from '../../loading-component/loading.component'
@@ -48,14 +36,12 @@ import { LoadingComponentWithPermissions } from '../../loading-component/loading
    PageHeaderComponent,
    ConfirmButtonComponent,
    NumberComponent,
-    SelectComponent,
    TextComponent,
    IfPermissionsDirective,
    DragDropSelectComponent,
    FormsModule,
    ReactiveFormsModule,
    AsyncPipe,
-    NgbPaginationModule,
    NgxBootstrapIconsModule,
  ],
 })
@@ -70,17 +56,8 @@ export class SavedViewsComponent
  private readonly modalService = inject(NgbModal)

  DisplayMode = DisplayMode
-  readonly savedViewIcons = SAVED_VIEW_ICONS

  readonly savedViews = signal<SavedView[]>(undefined)
-  readonly page = signal(1)
-  public readonly pageSize = 25
-  // All views are loaded at init, so paging is only for display
-  readonly pagedSavedViews = computed(() => {
-    const start = (this.page() - 1) * this.pageSize
-    return this.savedViews()?.slice(start, start + this.pageSize)
-  })
-
  private savedViewsGroup = new FormGroup({})
  public savedViewsForm: FormGroup = new FormGroup({
    savedViews: this.savedViewsGroup,
@@ -107,11 +84,9 @@ export class SavedViewsComponent
  private reloadViews(): void {
    this.loading.set(true)
    this.savedViewService
-      .list(1, 100000, null, false, { full_perms: true })
+      .list(null, null, null, false, { full_perms: true })
      .subscribe((r) => {
        this.savedViews.set(r.results)
-        const pageCount = Math.ceil(r.results.length / this.pageSize)
-        this.page.update((page) => Math.min(page, Math.max(1, pageCount)))
        this.initialize()
      })
  }
@@ -135,7 +110,6 @@ export class SavedViewsComponent
      storeData.savedViews[view.id.toString()] = {
        id: view.id,
        name: view.name,
-        icon: view.icon ?? DEFAULT_SAVED_VIEW_ICON,
        show_on_dashboard: view.show_on_dashboard,
        show_in_sidebar: view.show_in_sidebar,
        page_size: view.page_size,
@@ -148,7 +122,6 @@ export class SavedViewsComponent
        new FormGroup({
          id: new FormControl({ value: null, disabled: !canEdit }),
          name: new FormControl({ value: null, disabled: !canEdit }),
-          icon: new FormControl({ value: null, disabled: !canEdit }),
          show_on_dashboard: new FormControl({
            value: null,
            disabled: false,
@@ -227,7 +200,6 @@ export class SavedViewsComponent

      const modelFieldsChanged =
        group.get('name')?.dirty ||
-        group.get('icon')?.dirty ||
        group.get('page_size')?.dirty ||
        group.get('display_mode')?.dirty ||
        group.get('display_fields')?.dirty
@@ -1,89 +0,0 @@
-export const DEFAULT_SAVED_VIEW_ICON = 'funnel'
-
-export const SAVED_VIEW_ICONS = [
-  { id: 'archive', name: $localize`Archive`, icon: 'archive' },
-  { id: 'bank', name: $localize`Bank`, icon: 'bank' },
-  { id: 'basket', name: $localize`Basket`, icon: 'basket' },
-  { id: 'bell', name: $localize`Bell`, icon: 'bell' },
-  { id: 'bookmark', name: $localize`Bookmark`, icon: 'bookmark' },
-  { id: 'boxes', name: $localize`Boxes`, icon: 'boxes' },
-  { id: 'briefcase', name: $localize`Briefcase`, icon: 'briefcase' },
-  { id: 'building', name: $localize`Building`, icon: 'building' },
-  { id: 'calculator', name: $localize`Calculator`, icon: 'calculator' },
-  { id: 'calendar', name: $localize`Calendar`, icon: 'calendar' },
-  { id: 'camera', name: $localize`Camera`, icon: 'camera' },
-  {
-    id: 'card-checklist',
-    name: $localize`Checklist`,
-    icon: 'card-checklist',
-  },
-  { id: 'cash', name: $localize`Cash`, icon: 'cash' },
-  { id: 'chat-left-text', name: $localize`Chat`, icon: 'chat-left-text' },
-  { id: 'check-circle', name: $localize`Check`, icon: 'check-circle' },
-  { id: 'clipboard', name: $localize`Clipboard`, icon: 'clipboard' },
-  { id: 'clock-history', name: $localize`Clock`, icon: 'clock-history' },
-  { id: 'credit-card', name: $localize`Credit card`, icon: 'credit-card' },
-  { id: 'download', name: $localize`Download`, icon: 'download' },
-  { id: 'envelope', name: $localize`Envelope`, icon: 'envelope' },
-  {
-    id: 'exclamation-triangle',
-    name: $localize`Warning`,
-    icon: 'exclamation-triangle',
-  },
-  { id: 'file-earmark', name: $localize`File`, icon: 'file-earmark' },
-  {
-    id: 'file-earmark-check',
-    name: $localize`Checked file`,
-    icon: 'file-earmark-check',
-  },
-  {
-    id: 'file-earmark-lock',
-    name: $localize`Locked file`,
-    icon: 'file-earmark-lock',
-  },
-  {
-    id: 'file-earmark-medical',
-    name: $localize`Medical file`,
-    icon: 'file-earmark-medical',
-  },
-  {
-    id: 'file-earmark-person',
-    name: $localize`Person file`,
-    icon: 'file-earmark-person',
-  },
-  {
-    id: 'file-earmark-spreadsheet',
-    name: $localize`Spreadsheet`,
-    icon: 'file-earmark-spreadsheet',
-  },
-  { id: 'file-text', name: $localize`Text file`, icon: 'file-text' },
-  { id: 'files', name: $localize`Files`, icon: 'files' },
-  { id: 'folder', name: $localize`Folder`, icon: 'folder' },
-  { id: 'funnel', name: $localize`Filter`, icon: 'funnel' },
-  { id: 'gear', name: $localize`Gear`, icon: 'gear' },
-  { id: 'globe2', name: $localize`Globe`, icon: 'globe2' },
-  { id: 'hash', name: $localize`Hash`, icon: 'hash' },
-  { id: 'heart', name: $localize`Heart`, icon: 'heart' },
-  { id: 'house', name: $localize`House`, icon: 'house' },
-  { id: 'inbox', name: $localize`Inbox`, icon: 'inbox' },
-  { id: 'journals', name: $localize`Journals`, icon: 'journals' },
-  { id: 'list-task', name: $localize`Task list`, icon: 'list-task' },
-  { id: 'newspaper', name: $localize`Newspaper`, icon: 'newspaper' },
-  { id: 'paperclip', name: $localize`Attachment`, icon: 'paperclip' },
-  { id: 'people', name: $localize`People`, icon: 'people' },
-  { id: 'person', name: $localize`Person`, icon: 'person' },
-  { id: 'printer', name: $localize`Printer`, icon: 'printer' },
-  { id: 'receipt', name: $localize`Receipt`, icon: 'receipt' },
-  { id: 'safe', name: $localize`Safe`, icon: 'safe' },
-  { id: 'search', name: $localize`Search`, icon: 'search' },
-  { id: 'send', name: $localize`Send`, icon: 'send' },
-  { id: 'shop', name: $localize`Shop`, icon: 'shop' },
-  { id: 'stack', name: $localize`Stack`, icon: 'stack' },
-  { id: 'stars', name: $localize`Stars`, icon: 'stars' },
-  { id: 'tag', name: $localize`Tag`, icon: 'tag' },
-  { id: 'tags', name: $localize`Tags`, icon: 'tags' },
-  { id: 'telephone', name: $localize`Telephone`, icon: 'telephone' },
-  { id: 'truck', name: $localize`Truck`, icon: 'truck' },
-  { id: 'upc-scan', name: $localize`Barcode`, icon: 'upc-scan' },
-  { id: 'wallet2', name: $localize`Wallet`, icon: 'wallet2' },
-]
@@ -5,8 +5,6 @@ import { ObjectWithPermissions } from './object-with-permissions'
 export interface SavedView extends ObjectWithPermissions {
  name?: string

-  icon?: string
-
  show_on_dashboard?: boolean

  show_in_sidebar?: boolean
@@ -35,27 +35,19 @@ import {
  arrowRightShort,
  arrowUpRight,
  asterisk,
-  bank,
-  basket,
  bell,
  bodyText,
-  bookmark,
  boxArrowUp,
  boxArrowUpRight,
  boxes,
  braces,
-  briefcase,
-  building,
-  calculator,
  calendar,
  calendarEvent,
  calendarEventFill,
-  camera,
  cardChecklist,
  cardHeading,
  caretDown,
  caretUp,
-  cash,
  chatLeftText,
  chatSquareDots,
  check,
@@ -73,7 +65,6 @@ import {
  clipboardCheckFill,
  clipboardFill,
  clockHistory,
-  creditCard,
  dash,
  dashCircle,
  diagram3,
@@ -92,12 +83,9 @@ import {
  fileEarmarkDiff,
  fileEarmarkFill,
  fileEarmarkLock,
-  fileEarmarkMedical,
  fileEarmarkMinus,
-  fileEarmarkPerson,
  fileEarmarkPlus,
  fileEarmarkRichtext,
-  fileEarmarkSpreadsheet,
  fileText,
  files,
  filter,
@@ -105,15 +93,12 @@ import {
  folderFill,
  funnel,
  gear,
-  globe2,
  google,
  grid,
  gripVertical,
  hash,
  hddStack,
-  heart,
  house,
-  inbox,
  infoCircle,
  journals,
  link,
@@ -121,9 +106,7 @@ import {
  listTask,
  listUl,
  microsoft,
-  newspaper,
  nodePlus,
-  paperclip,
  pencil,
  people,
  peopleFill,
@@ -138,12 +121,9 @@ import {
  plusCircle,
  printer,
  questionCircle,
-  receipt,
-  safe,
  scissors,
  search,
  send,
-  shop,
  slashCircle,
  sliders2Vertical,
  sortAlphaDown,
@@ -153,17 +133,14 @@ import {
  tag,
  tagFill,
  tags,
-  telephone,
  textIndentLeft,
  textLeft,
  threeDots,
  threeDotsVertical,
  trash,
-  truck,
  uiRadios,
  unlock,
  upcScan,
-  wallet2,
  windowStack,
  x,
  xCircle,
@@ -281,22 +258,15 @@ const icons = {
  arrowRightShort,
  arrowUpRight,
  asterisk,
-  bank,
-  basket,
  bell,
  braces,
  bodyText,
-  bookmark,
  boxArrowUp,
  boxArrowUpRight,
  boxes,
-  briefcase,
-  building,
-  calculator,
  calendar,
  calendarEvent,
  calendarEventFill,
-  camera,
  cardChecklist,
  cardHeading,
  caretDown,
@@ -318,8 +288,6 @@ const icons = {
  clipboardCheckFill,
  clipboardFill,
  clockHistory,
-  cash,
-  creditCard,
  dash,
  dashCircle,
  diagram3,
@@ -338,12 +306,9 @@ const icons = {
  fileEarmarkDiff,
  fileEarmarkFill,
  fileEarmarkLock,
-  fileEarmarkMedical,
  fileEarmarkMinus,
-  fileEarmarkPerson,
  fileEarmarkPlus,
  fileEarmarkRichtext,
-  fileEarmarkSpreadsheet,
  files,
  fileText,
  filter,
@@ -351,15 +316,12 @@ const icons = {
  folderFill,
  funnel,
  gear,
-  globe2,
  google,
  grid,
  gripVertical,
  hash,
  hddStack,
-  heart,
  house,
-  inbox,
  infoCircle,
  journals,
  link,
@@ -367,10 +329,8 @@ const icons = {
  listTask,
  listUl,
  microsoft,
-  newspaper,
  nodePlus,
  pencil,
-  paperclip,
  people,
  peopleFill,
  person,
@@ -384,13 +344,10 @@ const icons = {
  plusCircle,
  printer,
  questionCircle,
-  receipt,
-  safe,
  scissors,
  search,
  send,
  slashCircle,
-  shop,
  sliders2Vertical,
  sortAlphaDown,
  sortAlphaUpAlt,
@@ -401,15 +358,12 @@ const icons = {
  tags,
  textIndentLeft,
  textLeft,
-  telephone,
  threeDots,
  threeDotsVertical,
  trash,
-  truck,
  uiRadios,
  unlock,
  upcScan,
-  wallet2,
  windowStack,
  x,
  xCircle,
@@ -55,7 +55,7 @@ class Command(PaperlessCommand):
            "--ratio",
            default=85.0,
            type=float,
-            help="Ratio to consider documents a match (0.0 - 100.0)",
+            help="Ratio to consider documents a match",
        )
        parser.add_argument(
            "--delete",
@@ -69,17 +69,6 @@ class Command(PaperlessCommand):
            action="store_true",
            help="Skip the confirmation prompt when used with --delete",
        )
-        parser.add_argument(
-            "--url",
-            default=None,
-            type=str,
-            help=(
-                "Base URL of the Paperless instance (e.g. "
-                "http://localhost:8000 or https://paperless.local). If set, matched "
-                "documents are shown as clickable (usually ctrl+click) links to "
-                "<url>/documents/<id>/details instead of by title."
-            ),
-        )

    def _render_results(
        self,
@@ -87,7 +76,6 @@ class Command(PaperlessCommand):
        *,
        opt_ratio: float,
        do_delete: bool,
-        base_url: str | None = None,
    ) -> list[int]:
        """Render match results as a Rich table. Returns list of PKs to delete."""
        if not matches:
@@ -100,22 +88,13 @@ class Command(PaperlessCommand):
            )
            return []

-        # Fetch titles for matched documents in a single query, unless we're
-        # going to show URLs instead.
-        titles: dict[int, str] = {}
-        if not base_url:
-            all_pks = {pk for m in matches for pk in (m.doc_one_pk, m.doc_two_pk)}
-            titles = dict(
-                Document.objects.filter(pk__in=all_pks)
-                .only("pk", "title")
-                .values_list("pk", "title"),
-            )
-
-        def _cell(pk: int) -> str:
-            if base_url:
-                doc_url = f"{base_url.rstrip('/')}/documents/{pk}/details"
-                return f"[link={doc_url}]{doc_url}[/link]"
-            return f"[dim]#{pk}[/dim] {titles.get(pk, 'Unknown')}"
+        # Fetch titles for matched documents in a single query.
+        all_pks = {pk for m in matches for pk in (m.doc_one_pk, m.doc_two_pk)}
+        titles: dict[int, str] = dict(
+            Document.objects.filter(pk__in=all_pks)
+            .only("pk", "title")
+            .values_list("pk", "title"),
+        )

        table = Table(
            title=f"Fuzzy Matches (threshold: {opt_ratio:.1f}%)",
@@ -145,8 +124,8 @@ class Command(PaperlessCommand):

            table.add_row(
                str(i),
-                _cell(pk_a),
-                _cell(pk_b),
+                f"[dim]#{pk_a}[/dim] {titles.get(pk_a, 'Unknown')}",
+                f"[dim]#{pk_b}[/dim] {titles.get(pk_b, 'Unknown')}",
                Text(f"{ratio:.1f}%", style=ratio_style),
            )
            maybe_delete_ids.append(pk_b)
@@ -229,7 +208,6 @@ class Command(PaperlessCommand):
            matches,
            opt_ratio=opt_ratio,
            do_delete=options["delete"],
-            base_url=options["url"],
        )

        if options["delete"] and maybe_delete_ids:
@@ -1,79 +0,0 @@
-from django.db import migrations
-from django.db import models
-
-
-class Migration(migrations.Migration):
-    dependencies = [
-        ("documents", "0022_add_perf_indexes"),
-    ]
-
-    operations = [
-        migrations.AddField(
-            model_name="savedview",
-            name="icon",
-            field=models.CharField(
-                choices=[
-                    ("archive", "Archive"),
-                    ("bank", "Bank"),
-                    ("basket", "Basket"),
-                    ("bell", "Bell"),
-                    ("bookmark", "Bookmark"),
-                    ("boxes", "Boxes"),
-                    ("briefcase", "Briefcase"),
-                    ("building", "Building"),
-                    ("calculator", "Calculator"),
-                    ("calendar", "Calendar"),
-                    ("camera", "Camera"),
-                    ("card-checklist", "Checklist"),
-                    ("cash", "Cash"),
-                    ("chat-left-text", "Chat"),
-                    ("check-circle", "Check"),
-                    ("clipboard", "Clipboard"),
-                    ("clock-history", "Clock"),
-                    ("credit-card", "Credit card"),
-                    ("download", "Download"),
-                    ("envelope", "Envelope"),
-                    ("exclamation-triangle", "Warning"),
-                    ("file-earmark", "File"),
-                    ("file-earmark-check", "Checked file"),
-                    ("file-earmark-lock", "Locked file"),
-                    ("file-earmark-medical", "Medical file"),
-                    ("file-earmark-person", "Person file"),
-                    ("file-earmark-spreadsheet", "Spreadsheet"),
-                    ("file-text", "Text file"),
-                    ("files", "Files"),
-                    ("folder", "Folder"),
-                    ("funnel", "Filter"),
-                    ("gear", "Gear"),
-                    ("globe2", "Globe"),
-                    ("hash", "Hash"),
-                    ("heart", "Heart"),
-                    ("house", "House"),
-                    ("inbox", "Inbox"),
-                    ("journals", "Journals"),
-                    ("list-task", "Task list"),
-                    ("newspaper", "Newspaper"),
-                    ("paperclip", "Attachment"),
-                    ("people", "People"),
-                    ("person", "Person"),
-                    ("printer", "Printer"),
-                    ("receipt", "Receipt"),
-                    ("safe", "Safe"),
-                    ("search", "Search"),
-                    ("send", "Send"),
-                    ("shop", "Shop"),
-                    ("stack", "Stack"),
-                    ("stars", "Stars"),
-                    ("tag", "Tag"),
-                    ("tags", "Tags"),
-                    ("telephone", "Telephone"),
-                    ("truck", "Truck"),
-                    ("upc-scan", "Barcode"),
-                    ("wallet2", "Wallet"),
-                ],
-                default="funnel",
-                max_length=64,
-                verbose_name="icon",
-            ),
-        ),
-    ]
@@ -519,68 +519,6 @@ class Document(SoftDeleteModel, ModelWithOwner):  # type: ignore[django-manager-


 class SavedView(ModelWithOwner):
-    class Icon(models.TextChoices):
-        ARCHIVE = ("archive", _("Archive"))
-        BANK = ("bank", _("Bank"))
-        BASKET = ("basket", _("Basket"))
-        BELL = ("bell", _("Bell"))
-        BOOKMARK = ("bookmark", _("Bookmark"))
-        BOXES = ("boxes", _("Boxes"))
-        BRIEFCASE = ("briefcase", _("Briefcase"))
-        BUILDING = ("building", _("Building"))
-        CALCULATOR = ("calculator", _("Calculator"))
-        CALENDAR = ("calendar", _("Calendar"))
-        CAMERA = ("camera", _("Camera"))
-        CARD_CHECKLIST = ("card-checklist", _("Checklist"))
-        CASH = ("cash", _("Cash"))
-        CHAT_LEFT_TEXT = ("chat-left-text", _("Chat"))
-        CHECK_CIRCLE = ("check-circle", _("Check"))
-        CLIPBOARD = ("clipboard", _("Clipboard"))
-        CLOCK_HISTORY = ("clock-history", _("Clock"))
-        CREDIT_CARD = ("credit-card", _("Credit card"))
-        DOWNLOAD = ("download", _("Download"))
-        ENVELOPE = ("envelope", _("Envelope"))
-        EXCLAMATION_TRIANGLE = ("exclamation-triangle", _("Warning"))
-        FILE_EARMARK = ("file-earmark", _("File"))
-        FILE_EARMARK_CHECK = ("file-earmark-check", _("Checked file"))
-        FILE_EARMARK_LOCK = ("file-earmark-lock", _("Locked file"))
-        FILE_EARMARK_MEDICAL = ("file-earmark-medical", _("Medical file"))
-        FILE_EARMARK_PERSON = ("file-earmark-person", _("Person file"))
-        FILE_EARMARK_SPREADSHEET = (
-            "file-earmark-spreadsheet",
-            _("Spreadsheet"),
-        )
-        FILE_TEXT = ("file-text", _("Text file"))
-        FILES = ("files", _("Files"))
-        FOLDER = ("folder", _("Folder"))
-        FUNNEL = ("funnel", _("Filter"))
-        GEAR = ("gear", _("Gear"))
-        GLOBE = ("globe2", _("Globe"))
-        HASH = ("hash", _("Hash"))
-        HEART = ("heart", _("Heart"))
-        HOUSE = ("house", _("House"))
-        INBOX = ("inbox", _("Inbox"))
-        JOURNALS = ("journals", _("Journals"))
-        LIST_TASK = ("list-task", _("Task list"))
-        NEWSPAPER = ("newspaper", _("Newspaper"))
-        PAPERCLIP = ("paperclip", _("Attachment"))
-        PEOPLE = ("people", _("People"))
-        PERSON = ("person", _("Person"))
-        PRINTER = ("printer", _("Printer"))
-        RECEIPT = ("receipt", _("Receipt"))
-        SAFE = ("safe", _("Safe"))
-        SEARCH = ("search", _("Search"))
-        SEND = ("send", _("Send"))
-        SHOP = ("shop", _("Shop"))
-        STACK = ("stack", _("Stack"))
-        STARS = ("stars", _("Stars"))
-        TAG = ("tag", _("Tag"))
-        TAGS = ("tags", _("Tags"))
-        TELEPHONE = ("telephone", _("Telephone"))
-        TRUCK = ("truck", _("Truck"))
-        UPC_SCAN = ("upc-scan", _("Barcode"))
-        WALLET = ("wallet2", _("Wallet"))
-
    class DisplayMode(models.TextChoices):
        TABLE = ("table", _("Table"))
        SMALL_CARDS = ("smallCards", _("Small Cards"))
@@ -603,13 +541,6 @@ class SavedView(ModelWithOwner):

    name = models.CharField(_("name"), max_length=128)

-    icon = models.CharField(
-        _("icon"),
-        max_length=64,
-        choices=Icon.choices,
-        default=Icon.FUNNEL,
-    )
-
    sort_field = models.CharField(
        _("sort field"),
        max_length=128,
@@ -1383,7 +1383,6 @@ class SavedViewSerializer(OwnedObjectSerializer):
        fields = [
            "id",
            "name",
-            "icon",
            "sort_field",
            "sort_reverse",
            "filter_rules",
@@ -3214,13 +3213,6 @@ class WorkflowActionSerializer(serializers.ModelSerializer[WorkflowAction]):
                        {"assign_title": f'Invalid f-string detected: "{e.args[0]}"'},
                    )

-        if attrs.get("assign_custom_fields_values"):
-            # Empty strings treated as None to avoid unexpected behavior
-            attrs["assign_custom_fields_values"] = {
-                field_id: (None if value == "" else value)
-                for field_id, value in attrs["assign_custom_fields_values"].items()
-            }
-
        if (
            "type" in attrs
            and attrs["type"] == WorkflowAction.WorkflowActionType.EMAIL
@@ -2905,20 +2905,18 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):

        v1 = SavedView.objects.get(name="test")
        self.assertEqual(v1.sort_field, "created2")
-        self.assertEqual(v1.icon, SavedView.Icon.FUNNEL)
        self.assertEqual(v1.filter_rules.count(), 1)
        self.assertEqual(v1.owner, self.user)

        response = self.client.patch(
            f"/api/saved_views/{v1.id}/",
-            {"sort_reverse": True, "icon": SavedView.Icon.RECEIPT},
+            {"sort_reverse": True},
            format="json",
        )

        v1 = SavedView.objects.get(id=v1.id)
        self.assertEqual(response.status_code, status.HTTP_200_OK)
        self.assertTrue(v1.sort_reverse)
-        self.assertEqual(v1.icon, SavedView.Icon.RECEIPT)
        self.assertEqual(v1.filter_rules.count(), 1)

        view["filter_rules"] = [{"rule_type": 12, "value": "secret"}]
@@ -2938,13 +2936,6 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
        v1 = SavedView.objects.get(id=v1.id)
        self.assertEqual(v1.filter_rules.count(), 0)

-        response = self.client.patch(
-            f"/api/saved_views/{v1.id}/",
-            {"icon": "not-an-icon"},
-            format="json",
-        )
-        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
-
    def test_saved_view_display_options(self) -> None:
        """
        GIVEN:
@@ -422,11 +422,6 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
            json.dumps(
                {
                    "assign_title": "",
-                    "assign_custom_fields": [self.cf1.id, self.cf2.id],
-                    "assign_custom_fields_values": {
-                        str(self.cf1.id): "",
-                        str(self.cf2.id): 0,
-                    },
                },
            ),
            content_type="application/json",
@@ -434,10 +429,6 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
        action = WorkflowAction.objects.get(id=response.data["id"])
        self.assertIsNone(action.assign_title)
-        self.assertEqual(
-            action.assign_custom_fields_values,
-            {str(self.cf1.id): None, str(self.cf2.id): 0},
-        )

        response = self.client.post(
            self.ENDPOINT_TRIGGERS,
@@ -1,4 +1,3 @@
-import os
 from io import StringIO
 from unittest.mock import patch

@@ -42,7 +41,7 @@ class TestFuzzyMatchCommand(TestCase):

    def test_invalid_ratio_upper_limit(self) -> None:
        """
-        GIVEN:
+        GIVEN:s
            - Invalid ratio above upper
        WHEN:
            - Command is called
@@ -109,45 +108,6 @@ class TestFuzzyMatchCommand(TestCase):
        stdout, _ = self.call_command("--processes", "1")
        self.assertIn("Found 1 matching pair(s)", stdout)

-    def test_with_matches_and_url(self) -> None:
-        """
-        GIVEN:
-            - 2 documents exist
-            - Similarity between content is 86.667
-            - --url is provided
-        WHEN:
-            - Command is called with --url
-        THEN:
-            - 1 match is returned from doc 1 to doc 2
-            - No match from doc 2 to doc 1 reported
-            - Output contains clickable links to the documents instead of titles
-        """
-        # Content similarity is 86.667
-        Document.objects.create(
-            checksum="BEEFCAFE",
-            title="A",
-            content="first document scanned by bob",
-            mime_type="application/pdf",
-            filename="test.pdf",
-        )
-        Document.objects.create(
-            checksum="DEADBEAF",
-            title="A",
-            content="first document scanned by alice",
-            mime_type="application/pdf",
-            filename="other_test.pdf",
-        )
-        with patch.dict(os.environ, {"COLUMNS": "200"}):
-            stdout, _ = self.call_command(
-                "--processes",
-                "1",
-                "--url",
-                "http://localhost:8000",
-            )
-        self.assertIn("Found 1 matching pair(s)", stdout)
-        self.assertIn("http://localhost:8000/documents/1/details", stdout)
-        self.assertIn("http://localhost:8000/documents/2/details", stdout)
-
    def test_with_3_matches(self) -> None:
        """
        GIVEN:
@@ -2000,55 +2000,6 @@ class TestWorkflows(
                r"Doc added in \w{3,}",
            )  # Match any 3-letter month name

-    def test_document_updated_workflow_existing_custom_field_empty_value(self) -> None:
-        """
-        GIVEN:
-            - Existing workflow with UPDATED trigger and action that assigns a custom field
-              with an empty value
-        WHEN:
-            - Document is updated that already contains the field with a value
-        THEN:
-            - The existing value is left untouched, see GH #13627
-        """
-        trigger = WorkflowTrigger.objects.create(
-            type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
-            filter_has_document_type=self.dt,
-        )
-        action = WorkflowAction.objects.create()
-        action.assign_custom_fields.add(self.cf1)
-        action.assign_custom_fields_values = {self.cf1.pk: ""}
-        action.save()
-        w = Workflow.objects.create(
-            name="Workflow 1",
-            order=0,
-        )
-        w.triggers.add(trigger)
-        w.actions.add(action)
-        w.save()
-
-        doc = Document.objects.create(
-            title="sample test",
-            correspondent=self.c,
-            original_filename="sample.pdf",
-        )
-        CustomFieldInstance.objects.create(
-            document=doc,
-            field=self.cf1,
-            value_text="existing value",
-        )
-
-        superuser = User.objects.create_superuser("superuser")
-        self.client.force_authenticate(user=superuser)
-
-        self.client.patch(
-            f"/api/documents/{doc.id}/",
-            {"document_type": self.dt.id},
-            format="json",
-        )
-
-        doc.refresh_from_db()
-        self.assertEqual(doc.custom_fields.get(field=self.cf1).value, "existing value")
-
    def test_document_updated_workflow_existing_custom_field(self) -> None:
        """
        GIVEN:
@@ -2267,7 +2267,7 @@ class ChatStreamingView(GenericAPIView[Any]):
            if not has_perms_owner_aware(request.user, "view_document", document):
                return HttpResponseForbidden("Insufficient permissions")

-            documents = Document.objects.filter(pk=document.pk)
+            documents = [document]
        else:
            documents = Document.objects.filter(
                id__in=permitted_document_ids(request.user),
@@ -105,8 +105,7 @@ def apply_assignment_to_document(
                field=field,
                document=document,
            ).first()
-            # empty string is indistinguishable from no value in the UI
-            if instance and args[value_field_name] not in (None, ""):
+            if instance and args[value_field_name] is not None:
                setattr(instance, value_field_name, args[value_field_name])
                instance.save()
            elif not instance:
@@ -3,9 +3,7 @@ Built-in remote-OCR document parser.

 Handles documents by sending them to a configured remote OCR engine
 (currently Azure AI Vision / Document Intelligence) and retrieving both
-the extracted text and a searchable PDF with an embedded text layer. For
-born-digital PDFs that need no archive copy, the remote call is skipped
-entirely in favor of locally-extracted text (see ``RemoteDocumentParser.parse``).
+the extracted text and a searchable PDF with an embedded text layer.

 When no engine is configured, ``score()`` returns ``None`` so the parser
 is effectively invisible to the registry — the tesseract parser handles
@@ -24,8 +22,6 @@ from typing import Self
 from django.conf import settings

 from documents.parsers import ParseError
-from paperless.parsers.utils import extract_pdf_text
-from paperless.parsers.utils import post_process_text
 from paperless.version import __full_version_str__

 if TYPE_CHECKING:
@@ -74,11 +70,8 @@ class RemoteDocumentParser:
    """Parse documents via a remote OCR API (currently Azure AI Vision).

    This parser sends documents to a remote engine that returns both
-    extracted text and a searchable PDF with an embedded text layer,
-    except when ``parse()`` is called with ``produce_archive=False`` for
-    a PDF, in which case the remote call is skipped and only locally
-    extracted text is returned (no archive). It does not depend on
-    Tesseract or ocrmypdf.
+    extracted text and a searchable PDF with an embedded text layer.
+    It does not depend on Tesseract or ocrmypdf.

    Class attributes
    ----------------
@@ -167,11 +160,8 @@ class RemoteDocumentParser:
        Returns
        -------
        bool
-            Always True — the remote engine is capable of returning a PDF
-            with an embedded text layer to serve as the archive copy.
-            Whether it actually does so for a given document depends on
-            ``produce_archive`` passed to :meth:`parse` (see there for when
-            the remote engine call, and thus archive generation, is skipped).
+            Always True — the remote engine always returns a PDF with an
+            embedded text layer that serves as the archive copy.
        """
        return True

@@ -228,12 +218,6 @@ class RemoteDocumentParser:
    ) -> None:
        """Send the document to the remote engine and store results.

-        When *produce_archive* is False for a PDF, the caller (via
-        ``documents.consumer.should_produce_archive``) has already determined
-        that the document is born-digital and needs no archive — skip the
-        remote engine entirely rather than re-OCRing it and creating a
-        duplicate text layer.
-
        Parameters
        ----------
        document_path:
@@ -241,8 +225,8 @@ class RemoteDocumentParser:
        mime_type:
            Detected MIME type of the document.
        produce_archive:
-            Whether an archive copy is wanted. For PDFs, False skips the
-            remote engine and uses locally-extracted text instead.
+            Ignored — the remote engine always returns a searchable PDF,
+            which is stored as the archive copy regardless of this flag.
        """
        config = RemoteEngineConfig(
            engine=settings.REMOTE_OCR_ENGINE,
@@ -257,16 +241,6 @@ class RemoteDocumentParser:
            self._text = ""
            return

-        if not produce_archive and mime_type == "application/pdf":
-            logger.debug(
-                "Remote OCR: skipped — no archive requested, "
-                "using locally-extracted text",
-            )
-            self._text = (
-                post_process_text(extract_pdf_text(document_path, log=logger)) or ""
-            )
-            return
-
        if config.engine == "azureai":
            self._text = self._azure_ai_vision_parse(document_path, config)

@@ -337,117 +337,6 @@ class TestRemoteParserParse:
        assert remote_parser.get_date() is None


-# ---------------------------------------------------------------------------
-# parse() — produce_archive=False skips the remote engine (PDFs only)
-# ---------------------------------------------------------------------------
-
-
-class TestRemoteParserSkipsWhenNoArchiveWanted:
-    """When the caller has already decided no archive is needed for a PDF
-    (documents.consumer.should_produce_archive), the remote engine call is
-    skipped entirely in favor of locally-extracted text.
-    """
-
-    def test_pdf_skips_azure_when_no_archive_requested(
-        self,
-        remote_parser: RemoteDocumentParser,
-        simple_digital_pdf_file: Path,
-        azure_client: Mock,
-    ) -> None:
-        """
-        GIVEN: produce_archive=False for a PDF
-        WHEN:  parse() is called
-        THEN:  Azure is never invoked, no archive is produced, and text
-               comes from local pdftotext extraction
-        """
-        remote_parser.parse(
-            simple_digital_pdf_file,
-            "application/pdf",
-            produce_archive=False,
-        )
-
-        azure_client.begin_analyze_document.assert_not_called()
-        assert remote_parser.get_archive_path() is None
-        assert remote_parser.get_text() != ""
-
-    def test_pdf_no_archive_requested_text_matches_local_extraction(
-        self,
-        remote_parser: RemoteDocumentParser,
-        simple_digital_pdf_file: Path,
-        azure_client: Mock,
-        mocker: MockerFixture,
-    ) -> None:
-        """
-        GIVEN: produce_archive=False for a PDF
-        WHEN:  parse() is called
-        THEN:  the returned text is exactly the locally-extracted text,
-               not anything from the (unused) Azure mock
-        """
-        mocker.patch(
-            "paperless.parsers.remote.extract_pdf_text",
-            return_value="Local digital text.",
-        )
-
-        remote_parser.parse(
-            simple_digital_pdf_file,
-            "application/pdf",
-            produce_archive=False,
-        )
-
-        assert remote_parser.get_text() == "Local digital text."
-
-    def test_pdf_no_archive_requested_closes_no_client(
-        self,
-        remote_parser: RemoteDocumentParser,
-        simple_digital_pdf_file: Path,
-        azure_client: Mock,
-    ) -> None:
-        remote_parser.parse(
-            simple_digital_pdf_file,
-            "application/pdf",
-            produce_archive=False,
-        )
-
-        azure_client.close.assert_not_called()
-
-    def test_non_pdf_still_calls_azure_when_no_archive_requested(
-        self,
-        remote_parser: RemoteDocumentParser,
-        simple_digital_pdf_file: Path,
-        azure_client: Mock,
-    ) -> None:
-        """
-        Images have no local-text fallback, so produce_archive=False does
-        not skip the remote engine for non-PDF MIME types.
-        """
-        remote_parser.parse(
-            simple_digital_pdf_file,
-            "image/png",
-            produce_archive=False,
-        )
-
-        azure_client.begin_analyze_document.assert_called_once()
-        assert remote_parser.get_text() == _DEFAULT_TEXT
-
-    @pytest.mark.usefixtures("no_engine_settings")
-    def test_unconfigured_engine_takes_precedence_over_skip(
-        self,
-        remote_parser: RemoteDocumentParser,
-        simple_digital_pdf_file: Path,
-    ) -> None:
-        """An unconfigured engine still short-circuits before the
-        produce_archive check, returning empty text as before.
-        """
-        remote_parser.parse(
-            simple_digital_pdf_file,
-            "application/pdf",
-            produce_archive=False,
-        )
-
-        assert remote_parser.get_text() == ""
-        assert remote_parser.get_archive_path() is None
-
-
 # ---------------------------------------------------------------------------
 # parse() — Azure failure path
 # ---------------------------------------------------------------------------
@@ -2,8 +2,6 @@ import json
 import logging
 import sys

-from django.db.models import QuerySet
-
 from documents.models import Document
 from paperless.config import AIConfig
 from paperless_ai.client import AIClient
@@ -84,21 +82,10 @@ def _build_document_reference(


 def _get_document_references(
-    documents: QuerySet[Document],
+    documents: list[Document],
    top_nodes: list,
 ) -> list[dict[str, int | str]]:
-    candidate_ids: set[int] = set()
-    for node in top_nodes:
-        try:
-            candidate_ids.add(int(node.metadata["document_id"]))
-        except (KeyError, TypeError, ValueError):  # pragma: no cover
-            continue
-
-    if not candidate_ids:
-        return []
-
-    allowed_documents = {doc.pk: doc for doc in documents.filter(pk__in=candidate_ids)}
-
+    allowed_documents = {doc.pk: doc for doc in documents}
    references: list[dict[str, int | str]] = []
    seen_document_ids: set[int] = set()

@@ -132,7 +119,7 @@ def _format_chat_metadata_trailer(references: list[dict[str, int | str]]) -> str

 def stream_chat_with_documents(
    query_str: str,
-    documents: QuerySet[Document],
+    documents: list[Document],
    output_language: str | None = None,
 ):
    try:
@@ -148,10 +135,10 @@ def stream_chat_with_documents(

 def _stream_chat_with_documents(
    query_str: str,
-    documents: QuerySet[Document],
+    documents: list[Document],
    output_language: str | None = None,
 ):
-    if not documents.exists():
+    if not documents:
        yield CHAT_NO_CONTENT_MESSAGE
        return

@@ -161,9 +148,7 @@ def _stream_chat_with_documents(
    from llama_index.core.retrievers import VectorIndexRetriever

    config = AIConfig()
-    filters = _document_id_filters(
-        str(pk) for pk in documents.values_list("pk", flat=True)
-    )
+    filters = _document_id_filters(str(doc.pk) for doc in documents)

    # Hold the shared read lock for the whole operation: the query engine
    # retrieves from the vector store again during synthesis, so the connection
@@ -24,7 +24,6 @@ from paperless_ai.embedding import get_configured_model_name
 from paperless_ai.embedding import get_embedding_model

 if TYPE_CHECKING:
-    from django.db.models import QuerySet
    from llama_index.core.schema import BaseNode

    from paperless_ai.vector_store import PaperlessSqliteVecVectorStore
@@ -253,20 +252,6 @@ def _safe_related_name(document: Document, field: str) -> str | None:
    return related.name if related else None


-def _document_index_queryset() -> "QuerySet[Document]":
-    """Document queryset with every relation build_document_node() /
-    build_llm_index_text() touches -- correspondent, document_type,
-    storage_path, tags, notes, custom_fields__field -- pre-loaded, so
-    indexing one document costs a fixed handful of queries regardless of
-    its tag or custom field count, instead of one query per related object.
-    """
-    return Document.objects.select_related(
-        "correspondent",
-        "document_type",
-        "storage_path",
-    ).prefetch_related("tags", "notes", "custom_fields__field")
-
-
 def build_document_node(
    document: Document,
    *,
@@ -440,7 +425,11 @@ def update_llm_index(
                "Skipping LLM index update: migration check deferred; "
                "will retry next run."
            )
-    documents = _document_index_queryset()
+    documents = Document.objects.select_related(
+        "correspondent",
+        "document_type",
+        "storage_path",
+    ).prefetch_related("tags", "notes", "custom_fields__field")
    no_documents = not documents.exists()

    # Fast exit before touching config: nothing to index and no existing index.
@@ -505,21 +494,6 @@ def update_llm_index(
 def llm_index_add_or_update_document(document: Document):
    """Add or atomically replace a document's chunks in the index."""
    config = AIConfig()
-    document_id = document.id
-    # Re-fetch with the same select_related/prefetch_related shape as the
-    # bulk path (update_llm_index()) uses: the caller's ``document`` instance
-    # (e.g. straight off a signal) has none of that loaded, and
-    # build_document_node()/build_llm_index_text() touch
-    # correspondent/document_type/storage_path/tags/notes/custom_fields__field
-    # -- without prefetching, that's one query per related object, including
-    # one per custom field instance.
-    document = _document_index_queryset().filter(pk=document_id).first()
-    if document is None:
-        logger.info(
-            "Skipping LLM index update for document %s: it no longer exists.",
-            document_id,
-        )
-        return
    new_nodes = build_document_node(
        document,
        chunk_size=config.llm_embedding_chunk_size,
@@ -735,71 +735,8 @@ class TestLlmIndexAddOrUpdateDocumentEmptyContent:
        )
        mock_load = mocker.patch("paperless_ai.indexing.load_or_build_index")

-        doc = DocumentFactory.create()
-        # Must not raise
-        indexing.llm_index_add_or_update_document(doc)
-
-        mock_load.assert_not_called()
-
-
-@pytest.mark.django_db
-class TestLlmIndexAddOrUpdateDocumentPrefetch:
-    """llm_index_add_or_update_document must prefetch the relations
-    build_document_node()/build_llm_index_text() touch, not re-query per
-    tag/note/custom field.
-    """
-
-    def test_query_count_does_not_scale_with_custom_field_count(
-        self,
-        temp_llm_index_dir: Path,
-        mock_embed_model: FakeEmbedding,
-    ) -> None:
-        """
-        GIVEN a document with several custom fields, a note, and a tag
-        WHEN it is incrementally indexed via llm_index_add_or_update_document
-        THEN the query count stays flat instead of growing with the number
-             of custom fields -- a regression here would add one query per
-             custom field instance (instance.field.name unprefetched), see
-             build_llm_index_text().
-        """
-        doc = DocumentFactory.create()
-        Note.objects.create(document=doc, note="a note")
-        for i in range(5):
-            field = CustomField.objects.create(
-                name=f"Field {i}",
-                data_type=CustomField.FieldDataType.STRING,
-            )
-            CustomFieldInstance.objects.create(
-                document=doc,
-                field=field,
-                value_text="value",
-            )
-
-        with CaptureQueriesContext(connection) as ctx:
-            indexing.llm_index_add_or_update_document(doc)
-
-        # Flat regardless of custom field count -- an unprefetched
-        # custom_fields__field would add one query per instance (5 here) on
-        # top of this budget.
-        assert len(ctx.captured_queries) <= 10
-
-    def test_skips_write_when_document_no_longer_exists(
-        self,
-        temp_llm_index_dir: Path,
-        mock_embed_model: FakeEmbedding,
-        mocker: pytest_mock.MockerFixture,
-    ) -> None:
-        """
-        GIVEN a document that has been deleted since the caller looked it up
-             (e.g. a race between a signal firing and its async task running)
-        WHEN llm_index_add_or_update_document is called with that stale instance
-        THEN it skips the write instead of raising Document.DoesNotExist
-        """
-        doc = DocumentFactory.create()
-        doc_id = doc.pk
-        Document.objects.filter(pk=doc_id).delete()
-        mock_load = mocker.patch("paperless_ai.indexing.load_or_build_index")
-
+        doc = MagicMock(spec=Document)
+        doc.id = 42
        # Must not raise
        indexing.llm_index_add_or_update_document(doc)

@@ -856,7 +793,8 @@ class TestLlmIndexLocking:
            return_value=[mock_node],
        )

-        doc = DocumentFactory.create()
+        doc = MagicMock(spec=Document)
+        doc.id = 1
        indexing.llm_index_add_or_update_document(doc)

        mock_store.upsert_document.assert_called_once()
@@ -887,7 +825,8 @@ class TestLlmIndexLocking:
            return_value=[mock_node],
        )

-        doc = DocumentFactory.create()
+        doc = MagicMock(spec=Document)
+        doc.id = 1
        indexing.llm_index_add_or_update_document(doc)

        mock_store.upsert_document.assert_not_called()
@@ -924,7 +863,8 @@ class TestLlmIndexLocking:
            return_value=[mock_node],
        )

-        doc = DocumentFactory.create()
+        doc = MagicMock(spec=Document)
+        doc.id = 1
        indexing.llm_index_add_or_update_document(doc)

        mock_store.upsert_document.assert_not_called()
@@ -3,12 +3,10 @@ from unittest.mock import MagicMock
 from unittest.mock import patch

 import pytest
-from django.db.models.signals import post_init
 from llama_index.core import settings as llama_settings
 from llama_index.core.embeddings.mock_embed_model import MockEmbedding
 from llama_index.core.schema import TextNode

-from documents.models import Document
 from documents.tests.factories import DocumentFactory
 from paperless_ai import chat
 from paperless_ai import indexing
@@ -38,6 +36,16 @@ def patch_embed_nodes():
        yield mock_embed_nodes


+@pytest.fixture
+def mock_document():
+    doc = MagicMock()
+    doc.pk = 1
+    doc.title = "Test Document"
+    doc.filename = "test_file.pdf"
+    doc.content = "This is the document content."
+    return doc
+
+
 def assert_chat_output(
    output: list[str],
    *,
@@ -53,13 +61,6 @@ def assert_chat_output(
    }


-def _fake_documents_queryset(pks: list[int]) -> MagicMock:
-    qs = MagicMock()
-    qs.exists.return_value = bool(pks)
-    qs.values_list.return_value = pks
-    return qs
-
-
@pytest.mark.parametrize(
    ("output_language", "expected_language_line"),
    [
@@ -106,10 +107,9 @@ def test_build_refine_prompt(

@pytest.mark.django_db
 def test_stream_chat_with_one_document_retrieval(
+    mock_document,
    patch_embed_nodes,
 ) -> None:
-    document = DocumentFactory.create(title="Test Document", content="ignored")
-    documents = Document.objects.filter(pk=document.pk)
    with (
        patch("paperless_ai.chat.AIClient") as mock_client_cls,
        patch("paperless_ai.chat.load_or_build_index") as mock_load_index,
@@ -124,19 +124,22 @@ def test_stream_chat_with_one_document_retrieval(
        mock_client_cls.return_value = mock_client
        mock_client.llm = MagicMock()

+        mock_node = TextNode(
+            text="This is node content.",
+            metadata={"document_id": str(mock_document.pk), "title": "Test Document"},
+        )
        mock_index = MagicMock()
-        mock_index.vector_store.get_nodes.return_value = [
-            TextNode(
-                text="This is node content.",
-                metadata={"document_id": str(document.pk), "title": "Test Document"},
-            ),
-        ]
+        # Simulate get_nodes returning nodes (content exists)
+        mock_index.vector_store.get_nodes.return_value = [mock_node]
        mock_load_index.return_value = mock_index

        mock_retriever_instance = MagicMock()
        mock_retriever_instance.retrieve.return_value = [
            MagicMock(
-                metadata={"document_id": str(document.pk), "title": "Test Document"},
+                metadata={
+                    "document_id": str(mock_document.pk),
+                    "title": "Test Document",
+                },
            ),
        ]

@@ -150,7 +153,7 @@ def test_stream_chat_with_one_document_retrieval(
            "llama_index.core.retrievers.VectorIndexRetriever",
            return_value=mock_retriever_instance,
        ):
-            output = list(stream_chat_with_documents("What is this?", documents))
+            output = list(stream_chat_with_documents("What is this?", [mock_document]))

        mock_query_engine.query.assert_called_once_with("What is this?")
        synthesizer_kwargs = mock_get_response_synthesizer.call_args.kwargs
@@ -163,16 +166,13 @@ def test_stream_chat_with_one_document_retrieval(
            output,
            expected_chunks=["chunk1", "chunk2"],
            expected_references=[
-                {"id": document.pk, "title": "Test Document"},
+                {"id": mock_document.pk, "title": "Test Document"},
            ],
        )


@pytest.mark.django_db
 def test_stream_chat_with_multiple_documents_retrieval(patch_embed_nodes) -> None:
-    doc1 = DocumentFactory.create(title="Document 1", content="ignored")
-    doc2 = DocumentFactory.create(title="Document 2", content="ignored")
-    documents = Document.objects.filter(pk__in=[doc1.pk, doc2.pk])
    with (
        patch("paperless_ai.chat.AIClient") as mock_client_cls,
        patch("paperless_ai.chat.load_or_build_index") as mock_load_index,
@@ -184,23 +184,23 @@ def test_stream_chat_with_multiple_documents_retrieval(patch_embed_nodes) -> Non
        mock_client_cls.return_value = mock_client
        mock_client.llm = MagicMock()

+        mock_node1 = TextNode(
+            text="Content for doc 1.",
+            metadata={"document_id": "1", "title": "Document 1"},
+        )
+        mock_node2 = TextNode(
+            text="Content for doc 2.",
+            metadata={"document_id": "2", "title": "Document 2"},
+        )
        mock_index = MagicMock()
-        mock_index.vector_store.get_nodes.return_value = [
-            TextNode(
-                text="Content for doc 1.",
-                metadata={"document_id": str(doc1.pk), "title": "Document 1"},
-            ),
-            TextNode(
-                text="Content for doc 2.",
-                metadata={"document_id": str(doc2.pk), "title": "Document 2"},
-            ),
-        ]
+        # Simulate get_nodes returning nodes (content exists)
+        mock_index.vector_store.get_nodes.return_value = [mock_node1, mock_node2]
        mock_load_index.return_value = mock_index

        mock_retriever_instance = MagicMock()
        mock_retriever_instance.retrieve.return_value = [
-            MagicMock(metadata={"document_id": str(doc1.pk), "title": "Document 1"}),
-            MagicMock(metadata={"document_id": str(doc2.pk), "title": "Document 2"}),
+            MagicMock(metadata={"document_id": "1", "title": "Document 1"}),
+            MagicMock(metadata={"document_id": "2", "title": "Document 2"}),
        ]

        mock_response_stream = MagicMock()
@@ -210,11 +210,14 @@ def test_stream_chat_with_multiple_documents_retrieval(patch_embed_nodes) -> Non
        mock_query_engine_cls.return_value = mock_query_engine
        mock_query_engine.query.return_value = mock_response_stream

+        doc1 = MagicMock(pk=1, title="Document 1", filename="doc1.pdf")
+        doc2 = MagicMock(pk=2, title="Document 2", filename="doc2.pdf")
+
        with patch(
            "llama_index.core.retrievers.VectorIndexRetriever",
            return_value=mock_retriever_instance,
        ):
-            output = list(stream_chat_with_documents("What's up?", documents))
+            output = list(stream_chat_with_documents("What's up?", [doc1, doc2]))

        mock_query_engine.query.assert_called_once_with("What's up?")
        patch_embed_nodes.assert_not_called()
@@ -222,15 +225,15 @@ def test_stream_chat_with_multiple_documents_retrieval(patch_embed_nodes) -> Non
            output,
            expected_chunks=["chunk1", "chunk2"],
            expected_references=[
-                {"id": doc1.pk, "title": "Document 1"},
-                {"id": doc2.pk, "title": "Document 2"},
+                {"id": 1, "title": "Document 1"},
+                {"id": 2, "title": "Document 2"},
            ],
        )


 def test_stream_chat_empty_document_list() -> None:
    with patch("paperless_ai.chat.load_or_build_index") as mock_load_index:
-        output = list(stream_chat_with_documents("Any info?", Document.objects.none()))
+        output = list(stream_chat_with_documents("Any info?", []))
        mock_load_index.assert_not_called()
        assert output == ["Sorry, I couldn't find any content to answer your question."]

@@ -250,9 +253,7 @@ def test_stream_chat_no_matching_nodes() -> None:
        mock_index.vector_store.get_nodes.return_value = []
        mock_load_index.return_value = mock_index

-        output = list(
-            stream_chat_with_documents("Any info?", _fake_documents_queryset([1])),
-        )
+        output = list(stream_chat_with_documents("Any info?", [MagicMock(pk=1)]))

        assert output == ["Sorry, I couldn't find any content to answer your question."]

@@ -281,9 +282,7 @@ def test_stream_chat_unexpected_failure_returns_generic_error(caplog) -> None:
            )
            mock_retriever_cls.return_value = mock_retriever

-            output = list(
-                stream_chat_with_documents("Any info?", _fake_documents_queryset([1])),
-            )
+            output = list(stream_chat_with_documents("Any info?", [MagicMock(pk=1)]))

        assert output == [CHAT_ERROR_MESSAGE]
        assert "Failed to stream document chat response" in caplog.text
@@ -299,12 +298,7 @@ class TestStreamChatRetrieval:
    ) -> None:
        doc = DocumentFactory.create(content="hello world")
        # Nothing indexed for this document yet.
-        out = list(
-            chat.stream_chat_with_documents(
-                "question?",
-                Document.objects.filter(pk=doc.pk),
-            ),
-        )
+        out = list(chat.stream_chat_with_documents("question?", [doc]))
        assert chat.CHAT_NO_CONTENT_MESSAGE in out

    def test_chat_filter_contains_only_requested_document_ids(
@@ -338,12 +332,7 @@ class TestStreamChatRetrieval:
            side_effect=capture_retriever,
        )

-        list(
-            chat.stream_chat_with_documents(
-                "question?",
-                Document.objects.filter(pk=included.pk),
-            ),
-        )
+        list(chat.stream_chat_with_documents("question?", [included]))

        assert captured_filters, "VectorIndexRetriever was never constructed"
        filt = captured_filters[0]
@@ -351,47 +340,3 @@ class TestStreamChatRetrieval:
        filter_values = filt.filters[0].value
        assert str(included.pk) in filter_values
        assert str(excluded.pk) not in filter_values
-
-    @pytest.mark.django_db
-    def test_get_document_references_only_queries_referenced_documents(
-        self,
-        django_assert_num_queries,
-    ) -> None:
-        """Building references must not hydrate every document the caller is
-        permitted to see -- only the (<= CHAT_RETRIEVER_TOP_K) documents that
-        the retriever actually returned nodes for.
-        """
-        referenced = DocumentFactory.create(title="Referenced Document")
-        # Many more documents are "accessible" but never referenced by a node.
-        DocumentFactory.create_batch(200)
-
-        documents = Document.objects.all()
-        top_nodes = [
-            MagicMock(
-                metadata={
-                    "document_id": str(referenced.pk),
-                    "title": "Referenced Document",
-                },
-            ),
-        ]
-
-        hydrated_count = 0
-
-        def _count_hydration(sender, instance, **kwargs):
-            nonlocal hydrated_count
-            hydrated_count += 1
-
-        post_init.connect(_count_hydration, sender=Document)
-        try:
-            # One query: `documents.filter(pk__in=candidate_ids)` for the single
-            # referenced id. No query should scale with the 200 unreferenced documents.
-            with django_assert_num_queries(1):
-                references = chat._get_document_references(documents, top_nodes)
-        finally:
-            post_init.disconnect(_count_hydration, sender=Document)
-
-        # The bug this guards against: the old code hydrated all 201 accessible
-        # documents via `{doc.pk: doc for doc in documents}` before filtering by
-        # top_nodes. Only the referenced document should ever be constructed.
-        assert hydrated_count == 1
-        assert references == [{"id": referenced.pk, "title": "Referenced Document"}]
Author	SHA1	Message	Date
stumpylog	3bd51461dc	docs: flag where the transition spec and plan describe a moved API The library changed after these were written and more changes are already decided upstream. Records what is wrong today, what to write toward, the one question still open, and the fast-JSON-field trap, rather than silently leaving code that would fail on contact.	2026-08-11 14:24:24 -07:00
stumpylog	812e784a68	chore: update transition guidance for the current whoosh-compat API Field references became a typed value rather than a dotted string, so diagnostics carry one too and the registry exposes a single resolver. Also records that the JSON fields must stay non-fast while existence checks against a fast JSON field return inverted results.	2026-08-11 14:21:52 -07:00
stumpylogandClaude Fable 5	c86edaa0e8	chore: build typed search errors from structured diagnostic data Diagnostics now carry field and raw_value, so the transition guidance points at those instead of parsing human-readable message text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-11 09:58:49 -07:00
stumpylogandClaude Sonnet 5	d8c9d22ea1	docs: add whoosh-compat transition implementation plan 16 bite-sized, TDD tasks across the design spec's 4-PR stack, each with a suggested subagent type/model for delegated execution. Test/fixture code in the acceptance-corpus and API-expansion tasks was verified against the real codebase (documents/tests/search/conftest.py's existing backend/index fixtures, test_backend.py's pytestmark convention, CustomFieldInstance's typed value_text field) rather than guessed, and the date-grammar parity test's AST-shape assumption was confirmed by actually running whoosh_compat.parse() against a real DATE FieldRegistry. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-09 21:27:46 -07:00
stumpylogandClaude Sonnet 5	50ed8c060b	docs: fold agent-review findings into whoosh-compat transition spec Agent review (source-verified against both repos) confirmed the spec's claims accurate throughout, with one real gap: the JSON-subpath tantivy-py carve-out (index.parse_query fallback for notes./ custom_fields. until tantivy-py#716 ships) interacts with paperless's pinned tantivy~=0.26.0 and wasn't mentioned. Also added two footnotes: FieldRegistry forces date_only=True on any DATE spec regardless of the PublicField default, and the date-grammar parity audit implicitly grants new keyword vocabulary as a side effect. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-09 21:27:46 -07:00
stumpylogandClaude Sonnet 5	00631146ff	docs: add whoosh-compat transition design spec Design for replacing _translate.py/_dates.py with whoosh-compat: shared field-definition table driving both the Tantivy schema and the query FieldRegistry, diagnostics->exception mapping (aggregating all errors, not just the first), a 4-PR stack with no rollout flag, and a result-level acceptance corpus + date-grammar parity audit as the safety net instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>	2026-08-09 21:27:46 -07:00
stumpylogandClaude Fable 5	d09caf480c	chore: add whoosh-compat transition skill Encodes the settled integration decisions for replacing the hand-maintained search translation layer with whoosh-compat: user-typed query surface policy, analyzer seam, diagnostics-before-emit contract, mandatory date parity audit, test churn, and rollout plan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-08-09 21:27:46 -07:00