The LLM-generated test for that last thing

* Ok! Backend stuff for the remote ocr workflow

* Frotnend workflow stuff

* And docs

* Fix dynamic action fields thing

* Actually, fix the action dropdown thing

* Fix this validation thing, and we have to check existing actions

* Fix migration

docs/api.md

@@ -302,7 +302,8 @@ The following methods are supported:
 - `delete`
   - No `parameters` required
 - `reprocess`
-  - No `parameters` required
+  - Optional `parameters`: `{ "remote_ocr": true }` to send the documents to the
+    remote OCR engine, see [Remote OCR](usage.md#remote-ocr). Defaults to false.
 - `set_permissions`
   - Requires `parameters`:
     - `"set_permissions": PERMISSIONS_OBJ` (see format [above](#permissions)) and / or

docs/configuration.md

@@ -2066,6 +2066,18 @@ password. All of these options come from their similarly-named [Django settings]
 
     Defaults to None.
 
+#### [`PAPERLESS_REMOTE_OCR_MODE=<str>`](#PAPERLESS_REMOTE_OCR_MODE) {#PAPERLESS_REMOTE_OCR_MODE}
+
+: Which documents are sent to the remote OCR engine.
+
+    - `always`: every document of a supported file type is sent to the remote
+      engine, bypassing the local OCR engine.
+    - `workflow_only`: documents are processed locally unless a workflow
+      explicitly enables remote OCR for them, letting you use the remote engine
+      selectively.
+
+    Defaults to "always".
+
 ## AI {#ai}
 
 #### [`PAPERLESS_AI_ENABLED=<bool>`](#PAPERLESS_AI_ENABLED) {#PAPERLESS_AI_ENABLED}

docs/development.md

@@ -456,6 +456,20 @@ def score(
     return 10
 ```
 
+**Remote services**
+
+If your parser sends document content to a remote service, declare it:
+
+```python
+class MyCustomParser:
+    uses_remote_service = True
+```
+
+Paperless-ngx excludes such parsers when the document being consumed has not
+been marked for remote processing, so users can keep remote OCR off by default
+and enable it selectively with a workflow. Parsers that do not declare the
+attribute are treated as fully local and are always considered.
+
 **Archive and rendition flags**
 
 ```python

docs/usage.md

@@ -654,6 +654,48 @@ happened while it was still encrypted, that original version will likewise be mi
 **Current limitation**: Passwords are stored as a simple list without descriptions. To handle
 multiple PDF types with different passwords, create separate workflows for each use case.
 
+##### Remote OCR {#workflow-action-remote-ocr}
+
+"Remote OCR" actions send the document to the configured remote OCR engine instead of processing it
+locally. To use remote OCR selectively, set the [remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE)
+to `workflow_only` then add this action to a workflow that matches only the documents you
+want sent to the remote engine. See [Remote OCR](#remote-ocr) for the engine setup. The action only works with
+a **Consumption Started** trigger.
+
+The action takes no options, its presence is what enables remote OCR for a matching document.
+
+If the remote engine is not configured, or does not support the document's file type, the document is
+processed locally instead and a warning is written to the log.
+
+##### Apply AI Suggestions {#workflow-action-apply-ai-suggestions}
+
+"Apply AI Suggestions" actions ask the configured AI service for title and metadata suggestions,
+the same as the AI suggestions shown on the document detail page, except applied automatically and in bulk.
+It requires [AI features](configuration.md#ai) to be enabled. You can specify:
+
+- Which suggestions to apply: title, tags, correspondent, document type, storage path and / or created
+  date. Suggestions for fields you did not select are discarded.
+- Whether to create missing items. By default only tags, correspondents and document types that
+  already exist are assigned and any other suggestion is dropped. With this enabled, suggested items
+  that do not exist are created. Storage paths are never created.
+- Whether to overwrite existing values. By default a field is only filled in if it is currently empty.
+  Note that documents almost always already have a title and created date, so if you select those you
+  will usually want to enable this too. Tags are an exception: suggested tags are always added and
+  never replace the document's existing tags.
+
+The action works with every trigger **except Consumption Started**, because suggestions are made from
+the document's text, which does not exist until after the document has been processed.
+
+Because the query to the AI service is slow, the action is queued and runs in the background rather
+than as part of the workflow run itself. The document is updated once the suggestions come back.
+
+!!! warning
+
+    Every matching document results in a query to the AI service, which may incur costs and have privacy
+    implications. Queries can be slow, so a workflow matching a large number of documents can occupy the
+    task queue, and delay consumption of new documents, etc. Consider narrowing the trigger filters,
+    running in small batches and / or increasing workers.
+
 #### Workflow placeholders
 
 Titles and webhook payloads can be generated by workflows using [Jinja templates](https://jinja.palletsprojects.com/en/3.1.x/templates/).
@@ -1090,11 +1132,19 @@ Paperless-ngx supports performing OCR on documents using remote services. At the
 [Microsoft's Azure "Document Intelligence" service](https://azure.microsoft.com/en-us/products/ai-services/ai-document-intelligence).
 This is of course a paid service (with a free tier) which requires an Azure account and subscription. Azure AI is not affiliated with
 Paperless-ngx in any way. When enabled, Paperless-ngx will automatically send appropriate documents to Azure for OCR processing, bypassing
-the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details.
+the local OCR engine. See the [configuration](configuration.md#PAPERLESS_REMOTE_OCR_ENGINE) options for more details. These
+settings can be supplied as environment variables or via **Application Configuration**.
 
 Additionally, when using a commercial service with this feature, consider both potential costs as well as any associated file size
 or page limitations (e.g. with a free tier).
 
+By default, every document of a supported file type is sent to the remote engine. To use it more selectively, set the
+[remote OCR mode](configuration.md#PAPERLESS_REMOTE_OCR_MODE) to `workflow_only`. Documents are then processed locally
+unless a [remote OCR workflow action](#workflow-action-remote-ocr) enables it for them, so you can limit the remote
+engine to particular documents.
+
+Setting the mode to `workflow_only` also allows the **Reprocess** actions to selectively use remote OCR for individual documents.
+
 ## Architecture
 
 Paperless-ngx consists of the following components:

pyproject.toml

@@ -77,7 +77,6 @@ dependencies = [
   "torch~=2.13.0",
   "watchfiles>=1.2",
   "whitenoise~=6.11",
-  "whoosh-compat[tantivy]==0.1",
   "zxing-cpp~=3.1.0",
 ]
 [project.optional-dependencies]

src-ui/src/app/components/admin/config/config.component.html

@@ -14,43 +14,48 @@
                 <a ngbNavLink>{{category}}</a>
                 <ng-template ngbNavContent>
                     <div class="p-3">
-                        <div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
-                            @for (option of getCategoryOptions(category); track option.key) {
-                                <div class="col">
-                                    <div class="card bg-light">
-                                        <div class="card-body">
-                                            <div class="card-title d-flex align-items-center">
-                                                <h6 class="mb-0">
-                                                  {{option.title}}
-                                                </h6>
-                                                <a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
-                                                    <i-bs name="info-circle"></i-bs>
-                                                </a>
-                                                @if (isSet(option.key)) {
-                                                  <button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
-                                                      <i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
-                                                  </button>
+                        @for (section of getCategorySections(category); track section) {
+                            @if (section) {
+                                <h5 class="mt-4 mb-3">{{section}}</h5>
+                            }
+                            <div class="row row-cols-1 row-cols-md-2 row-cols-lg-3 g-2">
+                                @for (option of getCategoryOptions(category, section); track option.key) {
+                                    <div class="col">
+                                        <div class="card bg-light">
+                                            <div class="card-body">
+                                                <div class="card-title d-flex align-items-center">
+                                                    <h6 class="mb-0">
+                                                      {{option.title}}
+                                                    </h6>
+                                                    <a class="btn btn-sm btn-link" title="Read the documentation about this setting" i18n-title [href]="getDocsUrl(option.config_key)" target="_blank" referrerpolicy="no-referrer">
+                                                        <i-bs name="info-circle"></i-bs>
+                                                    </a>
+                                                    @if (isSet(option.key)) {
+                                                      <button type="button" class="btn btn-sm btn-link text-danger ms-auto pe-0" title="Reset" i18n-title (click)="resetOption(option.key)">
+                                                          <i-bs class="me-1" name="x"></i-bs><ng-container i18n>Reset</ng-container>
+                                                      </button>
+                                                    }
+                                                </div>
+                                                <div class="mb-n3">
+                                                    @switch (option.type) {
+                                                        @case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
+                                                        @case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
+                                                        @case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
+                                                        @case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
+                                                        @case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
+                                                        @case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
+                                                        @case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
+                                                    }
+                                                </div>
+                                                @if (option.note) {
+                                                    <div class="form-text fst-italic">{{option.note}}</div>
                                                 }
                                             </div>
-                                            <div class="mb-n3">
-                                                @switch (option.type) {
-                                                    @case (ConfigOptionType.Select) { <pngx-input-select [formControlName]="option.key" [error]="errors[option.key]" [items]="option.choices" [allowNull]="true"></pngx-input-select> }
-                                                    @case (ConfigOptionType.Number) { <pngx-input-number [formControlName]="option.key" [error]="errors[option.key]" [showAdd]="false"></pngx-input-number> }
-                                                    @case (ConfigOptionType.Boolean) { <pngx-input-switch [formControlName]="option.key" [error]="errors[option.key]" [showUnsetNote]="true" [horizontal]="true" title="Enable" i18n-title></pngx-input-switch> }
-                                                    @case (ConfigOptionType.String) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
-                                                    @case (ConfigOptionType.JSON) { <pngx-input-text [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-text> }
-                                                    @case (ConfigOptionType.File) { <pngx-input-file [formControlName]="option.key" (upload)="uploadFile($event, option.key)" [error]="errors[option.key]"></pngx-input-file> }
-                                                    @case (ConfigOptionType.Password) { <pngx-input-password [formControlName]="option.key" [error]="errors[option.key]"></pngx-input-password> }
-                                                }
-                                            </div>
-                                            @if (option.note) {
-                                                <div class="form-text fst-italic">{{option.note}}</div>
-                                            }
                                         </div>
                                     </div>
-                                </div>
-                            }
-                        </div>
+                                }
+                            </div>
+                        }
                     </div>
                 </ng-template>
             </li>

src-ui/src/app/components/admin/config/config.component.spec.ts

@@ -8,7 +8,11 @@ import { NgbModule } from '@ng-bootstrap/ng-bootstrap'
 import { NgSelectModule } from '@ng-select/ng-select'
 import { NgxBootstrapIconsModule, allIcons } from 'ngx-bootstrap-icons'
 import { of, throwError } from 'rxjs'
-import { OutputTypeConfig } from 'src/app/data/paperless-config'
+import {
+  ConfigCategory,
+  ConfigSection,
+  OutputTypeConfig,
+} from 'src/app/data/paperless-config'
 import { ConfigService } from 'src/app/services/config.service'
 import { SettingsService } from 'src/app/services/settings.service'
 import { ToastService } from 'src/app/services/toast.service'
@@ -158,4 +162,24 @@ describe('ConfigComponent', () => {
     component.resetOption('barcodes_enabled')
     expect(component.configForm.get('barcodes_enabled').value).toBeNull()
   })
+
+  it('should group options into sections within a category, or not', () => {
+    const sections = component.getCategorySections(ConfigCategory.OCR)
+    expect(sections).toEqual([null, ConfigSection.RemoteOCR])
+    expect(
+      component
+        .getCategoryOptions(ConfigCategory.OCR)
+        .map((option) => option.key)
+    ).toContain('output_type')
+    expect(
+      component
+        .getCategoryOptions(ConfigCategory.OCR, ConfigSection.RemoteOCR)
+        .map((option) => option.key)
+    ).toEqual([
+      'remote_ocr_engine',
+      'remote_ocr_api_key',
+      'remote_ocr_endpoint',
+      'remote_ocr_mode',
+    ])
+  })
 })

src-ui/src/app/components/admin/config/config.component.ts

@@ -74,8 +74,20 @@ export class ConfigComponent
     return Object.values(ConfigCategory)
   }
 
-  getCategoryOptions(category: string): ConfigOption[] {
-    return PaperlessConfigOptions.filter((o) => o.category === category)
+  getCategorySections(category: string): string[] {
+    return [
+      ...new Set(
+        PaperlessConfigOptions.filter((o) => o.category === category).map(
+          (o) => o.section ?? null // null means no section
+        )
+      ),
+    ]
+  }
+
+  getCategoryOptions(category: string, section: string = null): ConfigOption[] {
+    return PaperlessConfigOptions.filter(
+      (o) => o.category === category && (o.section ?? null) === section
+    )
   }
 
   initialConfig: PaperlessConfig

src-ui/src/app/components/app-frame/app-frame.component.html

@@ -2,7 +2,7 @@
   <button class="navbar-toggler d-md-none collapsed border-0" type="button" data-toggle="collapse"
     data-target="#sidebarMenu" aria-controls="sidebarMenu" aria-expanded="false" aria-label="Toggle navigation"
     (click)="closeMobileSearch(); toggleMenuCollapsed()">
-    <span class="navbar-toggler-icon"></span>
+    <i-bs width="1.5em" height="1.5em" name="list"></i-bs>
   </button>
   <a class="navbar-brand d-flex align-items-center me-0 ps-md-3 py-0 order-sm-0"
     [ngClass]="{ 'slim': slimSidebarEnabled, '' : !slimSidebarEnabled }"
@@ -10,16 +10,12 @@
     tourAnchor="tour.intro">
     @if (!hasCustomBranding) {
       <pngx-logo extra_classes="navbar-official-logo px-1" height="2.4rem"></pngx-logo>
-      <svg class="brand-mark brand-mark-slim d-none" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
-        <path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
-      </svg>
+      <pngx-brand-mark class="brand-mark brand-mark-slim d-none"></pngx-brand-mark>
     } @else {
       @if (customAppLogo) {
         <img class="brand-logo" [src]="customAppLogo" alt="" />
       } @else {
-        <svg class="brand-mark" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" width="1.5em" height="1.5em" fill="currentColor">
-          <path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
-        </svg>
+        <pngx-brand-mark class="brand-mark"></pngx-brand-mark>
       }
       <div class="brand-copy ms-2 text-truncate" [class.d-md-none]="slimSidebarEnabled">
         <span class="brand-title text-truncate">{{ appTitle }}</span>
@@ -29,9 +25,9 @@
       </div>
     }
   </a>
-  <div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 px-3 ps-md-3 me-sm-auto order-3 order-sm-1"
+  <div class="search-container flex-grow-1 py-2 pb-3 pb-sm-2 me-sm-auto order-3 order-sm-1"
     [class.mobile-hidden]="mobileSearchHidden()">
-    <div class="col-12 header-search">
+    <div class="col-12 header-search mx-auto">
       <pngx-global-search></pngx-global-search>
     </div>
   </div>
@@ -90,7 +86,7 @@
         <ul class="nav flex-column">
           <li class="nav-item app-link">
             <a class="nav-link" routerLink="dashboard" routerLinkActive="active" (click)="closeMenu()"
-              ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+              ngbPopover="Dashboard" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
               container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
               <i-bs class="me-2" name="house"></i-bs><span><ng-container i18n>Dashboard</ng-container></span>
             </a>
@@ -99,7 +95,7 @@
             <a class="nav-link" routerLink="documents" routerLinkActive="active"
               [routerLinkActiveOptions]="{ paths: 'exact', queryParams: 'ignored', matrixParams: 'ignored', fragment: 'ignored' }"
               (click)="closeMenu()"
-              ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+              ngbPopover="Documents" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
               container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
               <i-bs class="me-2" name="files"></i-bs><span><ng-container i18n>Documents</ng-container></span>
             </a>
@@ -118,7 +114,7 @@
                   (cdkDragEnded)="onDragEnd($event)">
                   <a class="nav-link" routerLink="view/{{view.id}}"
                     routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="view.name"
-                    [disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
+                    [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
                     popoverClass="popover-slim">
                     <i-bs class="me-2" [name]="view.icon || 'funnel'"></i-bs><span><div class="d-inline-flex view-name"><span class="overflow-hidden" [class.text-wrap]="!slimSidebarEnabled">{{view.name}}</span></div>
                         @if (showSidebarCounts && !slimSidebarEnabled) {
@@ -156,7 +152,7 @@
               <li class="nav-item w-100 app-link">
                 <a class="nav-link app-link" [class.text-truncate]="!slimSidebarEnabled" routerLink="documents/{{d.id}}"
                   routerLinkActive="active" (click)="closeMenu()" [ngbPopover]="d.title | documentTitle"
-                  [disablePopover]="!slimSidebarEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
+                  [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body" triggers="mouseenter:mouseleave"
                   popoverClass="popover-slim">
                   <i-bs class="me-2" name="file-text"></i-bs><span>{{d.title | documentTitle}}</span>
                   <span class="close flex-column justify-content-center"
@@ -170,7 +166,7 @@
               <li class="nav-item w-100 app-link">
                 <button type="button" class="nav-link nav-link-action app-link w-100 text-start"
                   [class.text-truncate]="!slimSidebarEnabled" (click)="closeAll()"
-                  ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                  ngbPopover="Close all" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                   container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                   <i-bs class="me-2" name="x"></i-bs><span><ng-container i18n>Close all</ng-container></span>
                 </button>
@@ -189,7 +185,7 @@
                 <div class="d-flex align-items-center attributes-row">
                   <a class="nav-link flex-fill" routerLink="attributes" routerLinkActive="active"
                     [routerLinkActiveOptions]="{ exact: !(slimSidebarEnabled || attributesSectionsCollapsed) }" (click)="closeMenu()"
-                    ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                    ngbPopover="Attributes" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                     container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                     <i-bs name="stack"></i-bs><span class="ms-2"><ng-container i18n>Attributes</ng-container></span>
                   </a>
@@ -241,7 +237,7 @@
             }
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.SavedView }">
               <a class="nav-link" routerLink="savedviews" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Saved Views" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="window-stack"></i-bs><span><ng-container i18n>Saved Views</ng-container></span>
               </a>
@@ -250,7 +246,7 @@
               *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.Workflow }"
               tourAnchor="tour.workflows">
               <a class="nav-link" routerLink="workflows" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Workflows" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="boxes"></i-bs><span><ng-container i18n>Workflows</ng-container></span>
               </a>
@@ -258,14 +254,14 @@
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.MailAccount }"
               tourAnchor="tour.mail">
               <a class="nav-link" routerLink="mail" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Mail"
-                i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
+                i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                 triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="envelope"></i-bs><span><ng-container i18n>Mail</ng-container></span>
               </a>
             </li>
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Delete, type: PermissionType.Document }">
               <a class="nav-link" routerLink="trash" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Trash"
-                i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
+                i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                 triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="trash"></i-bs><span><ng-container i18n>Trash</ng-container></span>
               </a>
@@ -281,21 +277,21 @@
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.UISettings }"
               tourAnchor="tour.settings">
               <a class="nav-link" routerLink="settings" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Settings" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="gear"></i-bs><span><ng-container i18n>Settings</ng-container></span>
               </a>
             </li>
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.Change, type: PermissionType.AppConfig }">
               <a class="nav-link" routerLink="config" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Configuration" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="sliders2-vertical"></i-bs><span><ng-container i18n>Configuration</ng-container></span>
               </a>
             </li>
             <li class="nav-item app-link" *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.User }">
               <a class="nav-link" routerLink="usersgroups" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Users & Groups" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="people"></i-bs><span><ng-container i18n>Users & Groups</ng-container></span>
               </a>
@@ -304,7 +300,7 @@
               *pngxIfPermissions="{ action: PermissionAction.View, type: PermissionType.PaperlessTask }"
               tourAnchor="tour.file-tasks">
               <a class="nav-link" routerLink="tasks" routerLinkActive="active" (click)="closeMenu()"
-                ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end"
+                ngbPopover="Tasks" i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end"
                 container="body" triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="me-2" name="list-task"></i-bs><span><ng-container i18n>Tasks</ng-container>@if (tasksService.needsAttentionTasks.length > 0) {
                   <span><span class="badge bg-danger ms-2 d-inline">{{tasksService.needsAttentionTasks.length}}</span></span>
@@ -317,26 +313,26 @@
             @if (permissionsService.isAdmin()) {
               <li class="nav-item app-link">
                 <a class="nav-link" routerLink="logs" routerLinkActive="active" (click)="closeMenu()" ngbPopover="Logs"
-                  i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
+                  i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                   triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                   <i-bs class="me-2" name="text-left"></i-bs><span><ng-container i18n>Logs</ng-container></span>
                 </a>
               </li>
             }
             <li class="nav-item mt-2" tourAnchor="tour.outro">
-              <a class="px-3 py-2 text-muted small d-flex align-items-center flex-wrap text-decoration-none"
+              <a class="text-muted small d-flex align-items-center flex-wrap text-decoration-none nav-anchor"
                 target="_blank" rel="noopener noreferrer" href="https://docs.paperless-ngx.com" ngbPopover="Documentation"
-                i18n-ngbPopover [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
+                i18n-ngbPopover [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                 triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                 <i-bs class="d-flex me-2" name="question-circle"></i-bs><span><ng-container i18n>Documentation</ng-container></span>
               </a>
             </li>
             <li class="nav-item" [class.visually-hidden]="slimSidebarEnabled">
-              <div class="px-3 py-0 text-muted small d-flex align-items-center flex-wrap">
+              <div class="text-muted small d-flex align-items-center flex-wrap nav-label">
                 <div class="me-3">
                   <a class="text-muted text-decoration-none" target="_blank" rel="noopener noreferrer"
                     href="https://github.com/paperless-ngx/paperless-ngx" ngbPopover="GitHub" i18n-ngbPopover
-                    [disablePopover]="!slimSidebarEnabled" placement="end" container="body"
+                    [disablePopover]="!slimSidebarPopoversEnabled" placement="end" container="body"
                     triggers="mouseenter:mouseleave" popoverClass="popover-slim">
                     {{ versionString }}
                   </a>

src-ui/src/app/components/app-frame/app-frame.component.scss

@@ -67,6 +67,23 @@
   main.mobile-search-hidden {
     padding-top: 56px;
   }
+
+  .navbar-toggler {
+    padding-left: calc(12px - (1.5em * 2.5 / 16));
+  }
+}
+
+.search-container {
+  padding-left: 1rem;
+  padding-right: 1rem;
+}
+
+// Below sm the search gets its own full-width row, so line it up with main's content edge
+@media (max-width: 575.98px) {
+  .search-container {
+    padding-left: 12px;
+    padding-right: 12px;
+  }
 }
 
 main {
@@ -97,7 +114,7 @@ main {
 
     li.nav-item span.badge {
       display: inline-block;
-      margin-right: 2px;
+      margin-right: -4px;
     }
   }
 
@@ -141,18 +158,11 @@ main {
     }
   }
 
-  .sidebar.slim {
-    li.nav-item span.badge {
-      display: inline-block;
-      margin-right: 2px;
-    }
-  }
-
   .sidebar-slim-toggler {
     display: block;
     position: fixed;
     left: calc(var(--pngx-sidebar-width) - 12px);
-    top: 72px;
+    bottom: 16px;
     z-index: 996;
     --bs-btn-padding-x: 0.35rem;
     --bs-btn-padding-y: 0.125rem;
@@ -227,6 +237,10 @@ main {
   }
 }
 
+.sidebar .nav-anchor, .sidebar .nav-label {
+  padding: .25rem .7rem;
+}
+
 .attributes-row {
   border-radius: .55rem;
   margin: .1rem 0;
@@ -403,7 +417,7 @@ main {
 
 :host ::ng-deep .navbar-official-logo {
   .leaf {
-    fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 70%, var(--bs-primary)) !important;
+    fill: color-mix(in srgb, var(--pngx-primary-text-contrast) 85%, var(--bs-primary)) !important;
   }
 
   .text {
@@ -444,7 +458,7 @@ main {
 
 .header-search {
   width: 100%;
-  max-width: 44rem;
+  max-width: 55rem;
 }
 
 @media screen and (max-width: 575.98px) {
@@ -458,6 +472,7 @@ main {
 
   .navbar-toggler {
     grid-area: toggler;
+    color: var(--pngx-primary-text-contrast);
     text-align: left;
   }
 

src-ui/src/app/components/app-frame/app-frame.component.ts

@@ -45,6 +45,7 @@ import { TasksService } from 'src/app/services/tasks.service'
 import { ToastService } from 'src/app/services/toast.service'
 import { environment } from 'src/environments/environment'
 import { ChatComponent } from '../chat/chat/chat.component'
+import { BrandMarkComponent } from '../common/logo/brand-mark/brand-mark.component'
 import { LogoComponent } from '../common/logo/logo.component'
 import { ProfileEditDialogComponent } from '../common/profile-edit-dialog/profile-edit-dialog.component'
 import { DocumentDetailComponent } from '../document-detail/document-detail.component'
@@ -61,6 +62,7 @@ const SCROLL_THRESHOLD = 16
   imports: [
     GlobalSearchComponent,
     LogoComponent,
+    BrandMarkComponent,
     DocumentTitlePipe,
     IfPermissionsDirective,
     ToastsDropdownComponent,
@@ -278,6 +280,10 @@ export class AppFrameComponent
       })
   }
 
+  get slimSidebarPopoversEnabled(): boolean {
+    return this.slimSidebarEnabled && !this.isMobileViewport()
+  }
+
   get attributesSectionsCollapsed(): boolean {
     this.settingsService.trackChanges()
     return this.settingsService

src-ui/src/app/components/common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component.html

@@ -0,0 +1,28 @@
+<div class="modal-header">
+  <h4 class="modal-title" id="modal-basic-title">{{title}}</h4>
+  <button type="button" class="btn-close" aria-label="Close" (click)="cancel()">
+  </button>
+</div>
+<div class="modal-body">
+  @if (messageBold) {
+    <p class="text-break"><b>{{messageBold}}</b></p>
+  }
+  @if (message) {
+    <p class="mb-0 text-break" [innerHTML]="message"></p>
+  }
+  @if (showRemoteOcr) {
+    <div class="form-check mt-3">
+      <input class="form-check-input" type="checkbox" id="reprocessRemoteOcr" [(ngModel)]="remoteOcr" />
+      <label class="form-check-label" for="reprocessRemoteOcr" i18n>Use remote OCR</label>
+      <div class="form-text" i18n>Sends the document to the configured remote OCR service, which may incur costs.</div>
+    </div>
+  }
+</div>
+<div class="modal-footer">
+  <button type="button" class="btn" [class]="cancelBtnClass" (click)="cancel()" [disabled]="!buttonsEnabled">
+    <span class="d-inline-block" style="padding-bottom: 1px;">{{cancelBtnCaption}}</span>
+  </button>
+  <button type="button" class="btn" [class]="btnClass" (click)="confirm()" [disabled]="!confirmButtonEnabled || !buttonsEnabled">
+    {{btnCaption}}
+  </button>
+</div>

src-ui/src/app/components/common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component.spec.ts

@@ -0,0 +1,72 @@
+import { provideHttpClient, withInterceptorsFromDi } from '@angular/common/http'
+import { provideHttpClientTesting } from '@angular/common/http/testing'
+import { ComponentFixture, TestBed } from '@angular/core/testing'
+import { NgbActiveModal } from '@ng-bootstrap/ng-bootstrap'
+import { RemoteOCRModeConfig } from 'src/app/data/paperless-config'
+import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
+import { SettingsService } from 'src/app/services/settings.service'
+import { ReprocessConfirmDialogComponent } from './reprocess-confirm-dialog.component'
+
+describe('ReprocessConfirmDialogComponent', () => {
+  let component: ReprocessConfirmDialogComponent
+  let fixture: ComponentFixture<ReprocessConfirmDialogComponent>
+  let settingsService: SettingsService
+
+  const createComponent = (configured: boolean, mode: string) => {
+    settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, configured)
+    settingsService.set(SETTINGS_KEYS.REMOTE_OCR_MODE, mode)
+
+    fixture = TestBed.createComponent(ReprocessConfirmDialogComponent)
+    component = fixture.componentInstance
+    fixture.detectChanges()
+  }
+
+  beforeEach(async () => {
+    TestBed.configureTestingModule({
+      providers: [
+        NgbActiveModal,
+        provideHttpClient(withInterceptorsFromDi()),
+        provideHttpClientTesting(),
+      ],
+      imports: [ReprocessConfirmDialogComponent],
+    }).compileComponents()
+
+    settingsService = TestBed.inject(SettingsService)
+  })
+
+  it('should not request remote OCR by default', () => {
+    createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
+
+    expect(component.remoteOcr).toBeFalsy()
+  })
+
+  it('should not offer remote OCR when no engine is configured', () => {
+    createComponent(false, RemoteOCRModeConfig.WORKFLOW_ONLY)
+
+    expect(component.showRemoteOcr).toBeFalsy()
+    expect(
+      fixture.nativeElement.querySelector('#reprocessRemoteOcr')
+    ).toBeNull()
+  })
+
+  it('should not offer remote OCR when it already handles every document', () => {
+    createComponent(true, RemoteOCRModeConfig.ALWAYS)
+
+    expect(component.showRemoteOcr).toBeFalsy()
+    expect(
+      fixture.nativeElement.querySelector('#reprocessRemoteOcr')
+    ).toBeNull()
+  })
+
+  it('should offer remote OCR when configured and selective', () => {
+    createComponent(true, RemoteOCRModeConfig.WORKFLOW_ONLY)
+
+    expect(component.showRemoteOcr).toBeTruthy()
+    const checkbox = fixture.nativeElement.querySelector('#reprocessRemoteOcr')
+    expect(checkbox).not.toBeNull()
+
+    checkbox.click()
+    fixture.detectChanges()
+    expect(component.remoteOcr).toBeTruthy()
+  })
+})

src-ui/src/app/components/common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component.ts

@@ -0,0 +1,20 @@
+import { Component, inject } from '@angular/core'
+import { FormsModule } from '@angular/forms'
+import { SettingsService } from 'src/app/services/settings.service'
+import { ConfirmDialogComponent } from '../confirm-dialog.component'
+
+@Component({
+  selector: 'pngx-reprocess-confirm-dialog',
+  templateUrl: './reprocess-confirm-dialog.component.html',
+  imports: [FormsModule],
+})
+export class ReprocessConfirmDialogComponent extends ConfirmDialogComponent {
+  private settings = inject(SettingsService)
+
+  remoteOcr: boolean = false
+
+  public get showRemoteOcr(): boolean {
+    // Hidden when it is not configured, or when it already handles every document anyway.
+    return this.settings.remoteOCRIsSelectable
+  }
+}

src-ui/src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.html

@@ -455,6 +455,52 @@
           </div>
         </div>
       }
+      @case (WorkflowActionType.RemoteOcr) {
+        <div class="row">
+          <div class="col">
+            <p class="text-muted small" i18n>The document will be sent to the configured remote OCR service. May incur costs.</p>
+          </div>
+        </div>
+      }
+      @case (WorkflowActionType.ApplyAiSuggestions) {
+        <div class="row">
+          <div class="col">
+            <p class="text-muted small" i18n>The document will be sent to the configured AI service for suggestions. Consider costs and privacy.</p>
+            <pngx-input-select
+              i18n-title
+              title="Apply suggestions for"
+              [items]="aiSuggestionFieldOptions"
+              [multiple]="true"
+              formControlName="ai_suggestion_fields"
+              [error]="error?.actions?.[i]?.ai_suggestion_fields"
+              hint="Suggestions for fields that are not selected are discarded."
+              i18n-hint
+            ></pngx-input-select>
+          </div>
+        </div>
+        <div class="row">
+          <div class="col-md-6">
+            <pngx-input-switch
+              [horizontal]="true"
+              i18n-title
+              title="Create missing items"
+              formControlName="ai_create_missing"
+              hint="Create suggested tags, correspondents and document types that do not exist yet."
+              i18n-hint
+            ></pngx-input-switch>
+          </div>
+          <div class="col-md-6">
+            <pngx-input-switch
+              [horizontal]="true"
+              i18n-title
+              title="Overwrite existing values"
+              formControlName="ai_overwrite_existing"
+              hint="Apply suggestions even if the document already has a value. Tags are always added, never replaced."
+              i18n-hint
+            ></pngx-input-switch>
+          </div>
+        </div>
+      }
     }
   </div>
 </ng-template>

src-ui/src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.spec.ts

@@ -22,6 +22,7 @@ import {
 } from 'src/app/data/matching-model'
 import { Workflow } from 'src/app/data/workflow'
 import {
+  AISuggestionField,
   WorkflowAction,
   WorkflowActionType,
 } from 'src/app/data/workflow-action'
@@ -29,6 +30,7 @@ import {
   DocumentSource,
   WorkflowTriggerType,
 } from 'src/app/data/workflow-trigger'
+import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { IfOwnerDirective } from 'src/app/directives/if-owner.directive'
 import { IfPermissionsDirective } from 'src/app/directives/if-permissions.directive'
 import { CorrespondentService } from 'src/app/services/rest/correspondent.service'
@@ -48,6 +50,7 @@ import { TagsComponent } from '../../input/tags/tags.component'
 import { TextComponent } from '../../input/text/text.component'
 import { EditDialogMode } from '../edit-dialog.component'
 import {
+  AI_SUGGESTION_FIELD_OPTIONS,
   DOCUMENT_SOURCE_OPTIONS,
   SCHEDULE_DATE_FIELD_OPTIONS,
   TriggerFilterType,
@@ -224,7 +227,12 @@ describe('WorkflowEditDialogComponent', () => {
     ).toEqual('Document Added')
     expect(component.getTriggerTypeOptionName(null)).toEqual('')
     expect(component.sourceOptions).toEqual(DOCUMENT_SOURCE_OPTIONS)
-    expect(component.actionTypeOptions).toEqual(WORKFLOW_ACTION_OPTIONS)
+    // Remote OCR is absent until the workflow has a consumption trigger
+    expect(component.actionTypeOptions).toEqual(
+      WORKFLOW_ACTION_OPTIONS.filter(
+        (a) => a.id !== WorkflowActionType.RemoteOcr
+      )
+    )
     expect(
       component.getActionTypeOptionName(WorkflowActionType.Assignment)
     ).toEqual('Assignment')
@@ -233,14 +241,231 @@ describe('WorkflowEditDialogComponent', () => {
       SCHEDULE_DATE_FIELD_OPTIONS
     )
 
-    // Email disabled
+    // Email, remote OCR and AI all disabled
     jest.spyOn(settingsService, 'get').mockReturnValue(false)
     component.ngOnInit()
     expect(component.actionTypeOptions).toEqual(
-      WORKFLOW_ACTION_OPTIONS.filter((a) => a.id !== WorkflowActionType.Email)
+      WORKFLOW_ACTION_OPTIONS.filter(
+        (a) =>
+          a.id !== WorkflowActionType.Email &&
+          a.id !== WorkflowActionType.RemoteOcr &&
+          a.id !== WorkflowActionType.ApplyAiSuggestions
+      )
     )
   })
 
+  it('should offer remote OCR only for consumption workflows', () => {
+    jest.spyOn(settingsService, 'get').mockReturnValue(true)
+
+    // A consumption trigger makes the action reachable
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.Consumption }],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+    expect(component.actionTypeOptions.map((a) => a.id)).toContain(
+      WorkflowActionType.RemoteOcr
+    )
+
+    // Any other trigger type runs after the document has been parsed
+    component.object = {
+      name: 'Workflow 2',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.RemoteOcr
+    )
+  })
+
+  it('should offer remote OCR on a trigger added to a new workflow', () => {
+    jest.spyOn(settingsService, 'get').mockReturnValue(true)
+    component.ngOnInit()
+
+    // Nothing for the action to apply to yet
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.RemoteOcr
+    )
+
+    // addTrigger creates the form field with emitEvent false, so the options
+    // have to be computed on read rather than cached from valueChanges
+    component.addTrigger()
+    expect(component.actionTypeOptions.map((a) => a.id)).toContain(
+      WorkflowActionType.RemoteOcr
+    )
+
+    // Switching that trigger to a type that runs after parsing removes it
+    component.triggerFields
+      .at(0)
+      .get('type')
+      .setValue(WorkflowTriggerType.DocumentAdded)
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.RemoteOcr
+    )
+  })
+
+  it('should keep remote OCR listed when an action already uses it', () => {
+    jest.spyOn(settingsService, 'get').mockReturnValue(true)
+
+    // Otherwise changing the trigger would silently blank the selection
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
+      actions: [{ type: WorkflowActionType.RemoteOcr }],
+    } as Workflow
+    component.ngOnInit()
+
+    expect(component.actionTypeOptions.map((a) => a.id)).toContain(
+      WorkflowActionType.RemoteOcr
+    )
+  })
+
+  it('should not offer remote OCR when no engine is configured', () => {
+    jest
+      .spyOn(settingsService, 'get')
+      .mockImplementation((key) => key !== SETTINGS_KEYS.REMOTE_OCR_CONFIGURED)
+
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.Consumption }],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.RemoteOcr
+    )
+  })
+
+  it('should offer apply AI suggestions unless every trigger is consumption', () => {
+    jest.spyOn(settingsService, 'get').mockReturnValue(true)
+
+    // Consumption runs before the document has been parsed, so there would be
+    // no content to make suggestions from
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.Consumption }],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.ApplyAiSuggestions
+    )
+
+    // A second, usable trigger is enough
+    component.object = {
+      name: 'Workflow 2',
+      order: 0,
+      enabled: true,
+      triggers: [
+        { type: WorkflowTriggerType.Consumption },
+        { type: WorkflowTriggerType.DocumentAdded },
+      ],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+    expect(component.actionTypeOptions.map((a) => a.id)).toContain(
+      WorkflowActionType.ApplyAiSuggestions
+    )
+  })
+
+  it('should keep apply AI suggestions listed when an action already uses it', () => {
+    jest.spyOn(settingsService, 'get').mockReturnValue(true)
+
+    // Otherwise changing the trigger would silently blank the selection
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.Consumption }],
+      actions: [{ type: WorkflowActionType.ApplyAiSuggestions }],
+    } as Workflow
+    component.ngOnInit()
+
+    expect(component.actionTypeOptions.map((a) => a.id)).toContain(
+      WorkflowActionType.ApplyAiSuggestions
+    )
+  })
+
+  it('should not offer apply AI suggestions when AI is disabled', () => {
+    jest
+      .spyOn(settingsService, 'get')
+      .mockImplementation((key) => key !== SETTINGS_KEYS.AI_ENABLED)
+
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
+      actions: [],
+    } as Workflow
+    component.ngOnInit()
+
+    expect(component.actionTypeOptions.map((a) => a.id)).not.toContain(
+      WorkflowActionType.ApplyAiSuggestions
+    )
+  })
+
+  it('should create form fields for apply AI suggestions options', () => {
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
+      actions: [
+        {
+          type: WorkflowActionType.ApplyAiSuggestions,
+          ai_suggestion_fields: [
+            AISuggestionField.Title,
+            AISuggestionField.Tags,
+          ],
+          ai_create_missing: true,
+          ai_overwrite_existing: true,
+        },
+      ],
+    } as Workflow
+    component.ngOnInit()
+
+    const action = component.actionFields.at(0)
+    expect(action.get('ai_suggestion_fields').value).toEqual([
+      AISuggestionField.Title,
+      AISuggestionField.Tags,
+    ])
+    expect(action.get('ai_create_missing').value).toBeTruthy()
+    expect(action.get('ai_overwrite_existing').value).toBeTruthy()
+    expect(component.aiSuggestionFieldOptions).toEqual(
+      AI_SUGGESTION_FIELD_OPTIONS
+    )
+  })
+
+  it('should default apply AI suggestions options on a new action', () => {
+    component.object = {
+      name: 'Workflow 1',
+      order: 0,
+      enabled: true,
+      triggers: [{ type: WorkflowTriggerType.DocumentAdded }],
+      actions: [],
+    } as Workflow
+    component.addAction()
+
+    const action = component.actionFields.at(component.actionFields.length - 1)
+    expect(action.get('ai_suggestion_fields').value).toEqual([])
+    expect(action.get('ai_create_missing').value).toBeFalsy()
+    expect(action.get('ai_overwrite_existing').value).toBeFalsy()
+  })
+
   it('should support add and remove triggers and actions', () => {
     component.object = workflow
     component.addTrigger()

src-ui/src/app/components/common/edit-dialog/workflow-edit-dialog/workflow-edit-dialog.component.ts

@@ -30,6 +30,7 @@ import { StoragePath } from 'src/app/data/storage-path'
 import { SETTINGS_KEYS } from 'src/app/data/ui-settings'
 import { Workflow } from 'src/app/data/workflow'
 import {
+  AISuggestionField,
   WorkflowAction,
   WorkflowActionType,
 } from 'src/app/data/workflow-action'
@@ -148,6 +149,41 @@ export const WORKFLOW_ACTION_OPTIONS = [
     id: WorkflowActionType.MoveToTrash,
     name: $localize`Move to trash`,
   },
+  {
+    id: WorkflowActionType.RemoteOcr,
+    name: $localize`Remote OCR`,
+  },
+  {
+    id: WorkflowActionType.ApplyAiSuggestions,
+    name: $localize`Apply AI suggestions`,
+  },
+]
+
+export const AI_SUGGESTION_FIELD_OPTIONS = [
+  {
+    id: AISuggestionField.Title,
+    name: $localize`Title`,
+  },
+  {
+    id: AISuggestionField.Tags,
+    name: $localize`Tags`,
+  },
+  {
+    id: AISuggestionField.Correspondent,
+    name: $localize`Correspondent`,
+  },
+  {
+    id: AISuggestionField.DocumentType,
+    name: $localize`Document type`,
+  },
+  {
+    id: AISuggestionField.StoragePath,
+    name: $localize`Storage path`,
+  },
+  {
+    id: AISuggestionField.Created,
+    name: $localize`Created date`,
+  },
 ]
 
 export enum TriggerFilterType {
@@ -504,8 +540,6 @@ export class WorkflowEditDialogComponent
 
   expandedItem: number = null
 
-  readonly allowedActionTypes = signal([])
-
   private readonly triggerFilterOptionsMap = new WeakMap<
     FormArray,
     TriggerFilterOption[]
@@ -548,13 +582,58 @@ export class WorkflowEditDialogComponent
       this.checkRemovalActionFields.bind(this)
     )
     this.checkRemovalActionFields(this.objectForm.value)
-    this.allowedActionTypes.set(
-      this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)
-        ? WORKFLOW_ACTION_OPTIONS
-        : WORKFLOW_ACTION_OPTIONS.filter(
-            (a) => a.id !== WorkflowActionType.Email
-          )
-    )
+  }
+
+  private allowedActionTypes: typeof WORKFLOW_ACTION_OPTIONS = null
+
+  private getAllowedActionTypes() {
+    let allowed = WORKFLOW_ACTION_OPTIONS
+
+    if (!this.settingsService.get(SETTINGS_KEYS.EMAIL_ENABLED)) {
+      allowed = allowed.filter((a) => a.id !== WorkflowActionType.Email)
+    }
+
+    // Remote OCR is decided before the document is parsed, so it is only
+    // offered for workflows that run at consumption.
+    const formWorkflow: Workflow = this.objectForm?.value
+    const remoteOcrUsable =
+      this.settingsService.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
+      (formWorkflow?.triggers?.some(
+        (trigger) => trigger.type === WorkflowTriggerType.Consumption
+      ) ||
+        formWorkflow?.actions?.some(
+          (action) => action.type === WorkflowActionType.RemoteOcr
+        ))
+    if (!remoteOcrUsable) {
+      allowed = allowed.filter((a) => a.id !== WorkflowActionType.RemoteOcr)
+    }
+
+    // Only available after consumption. Unlike remote OCR this is hidden only
+    // once every trigger is consumption, so it stays offered on a workflow
+    // that has no triggers yet.
+    const aiSuggestionsUsable =
+      this.settingsService.get(SETTINGS_KEYS.AI_ENABLED) &&
+      (!formWorkflow?.triggers?.length ||
+        formWorkflow.triggers.some(
+          (trigger) => trigger.type !== WorkflowTriggerType.Consumption
+        ) ||
+        formWorkflow.actions?.some(
+          (action) => action.type === WorkflowActionType.ApplyAiSuggestions
+        ))
+    if (!aiSuggestionsUsable) {
+      allowed = allowed.filter(
+        (a) => a.id !== WorkflowActionType.ApplyAiSuggestions
+      )
+    }
+
+    if (
+      this.allowedActionTypes?.length === allowed.length &&
+      this.allowedActionTypes.every((a, i) => a.id === allowed[i].id)
+    ) {
+      return this.allowedActionTypes
+    }
+    this.allowedActionTypes = allowed
+    return allowed
   }
 
   private checkRemovalActionFields(formWorkflow: Workflow) {
@@ -1198,6 +1277,11 @@ export class WorkflowEditDialogComponent
         passwords: new FormControl(
           this.formatPasswords(action.passwords ?? [])
         ),
+        ai_suggestion_fields: new FormControl(
+          action.ai_suggestion_fields ?? []
+        ),
+        ai_create_missing: new FormControl(!!action.ai_create_missing),
+        ai_overwrite_existing: new FormControl(!!action.ai_overwrite_existing),
       }),
       { emitEvent }
     )
@@ -1279,13 +1363,18 @@ export class WorkflowEditDialogComponent
 
   get actionTypeOptions() {
     this.settingsService.trackChanges()
-    return this.allowedActionTypes()
+    // Computed on read rather than cached
+    return this.getAllowedActionTypes()
   }
 
   getActionTypeOptionName(type: WorkflowActionType): string {
     return this.actionTypeOptions.find((t) => t.id === type)?.name ?? ''
   }
 
+  get aiSuggestionFieldOptions() {
+    return AI_SUGGESTION_FIELD_OPTIONS
+  }
+
   addAction() {
     if (!this.object) {
       this.object = Object.assign({}, this.objectForm.value)
@@ -1339,6 +1428,9 @@ export class WorkflowEditDialogComponent
         include_document: false,
       },
       passwords: [],
+      ai_suggestion_fields: [],
+      ai_create_missing: false,
+      ai_overwrite_existing: false,
     }
     this.object.actions.push(action)
     this.createActionField(action)

src-ui/src/app/components/common/logo/brand-mark/brand-mark.component.html

@@ -0,0 +1,3 @@
+<svg class="w-100 h-100 d-block" xmlns="http://www.w3.org/2000/svg" viewBox="0 0 1000 1000" fill="currentColor" aria-hidden="true">
+  <path d="M341,949.1c-6.9-20.3-20.7-61.2-21.9-61-199.6-88.9-182.5-229.8-134.3-347.5,30,137.2,268.8,148.9,146.2,336-.9,2.2,10,27.8,19.5,51.3,22.7-51.9,58.6-115.5,55.8-120.8C178,398.7,724.9,299,807.1,18.5c83,251.5,53.1,659.8-377.4,814.9-2,1.4-63.5,148.6-66.9,150.2-.2-2.1-33.2,2.9-30.1-8.7,1.6-7,4.8-16.2,8.2-25.6h0v-.2h.1ZM323.1,846.2c48.3-71.9-12.7-120.8-56.9-152.2,81.2,107.4,66.4,120.8,56.9,152.2h0Z"/>
+</svg>

src-ui/src/app/components/common/logo/brand-mark/brand-mark.component.ts

@@ -0,0 +1,14 @@
+import { Component, input } from '@angular/core'
+
+@Component({
+  selector: 'pngx-brand-mark',
+  templateUrl: './brand-mark.component.html',
+  host: {
+    '[style.width]': 'width()',
+    '[style.height]': 'height()',
+  },
+})
+export class BrandMarkComponent {
+  readonly width = input<string>(null)
+  readonly height = input<string>(null)
+}

src-ui/src/app/components/common/page-header/page-header.component.html

@@ -1,6 +1,6 @@
-<div class="row pt-3 pb-3 pb-md-2 align-items-center">
+<div class="row pt-3 pb-2 align-items-center">
   <div class="col-md text-truncate">
-    <h3 class="d-flex align-items-center mb-1" style="line-height: 1.4">
+    <h3 class="d-flex align-items-center mb-2 mb-md-1" style="line-height: 1.4">
       <span class="text-truncate">{{title()}}</span>
       @if (id()) {
         <span class="badge bg-primary text-primary-text-contrast ms-3 small fs-normal cursor-pointer" (click)="copyID()">
@@ -12,10 +12,10 @@
         </span>
       }
       @if (subTitle()) {
-        <span class="h6 mb-0 mt-1 d-block d-md-inline fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
+        <span class="page-subtitle h6 mb-0 mt-1 fw-normal ms-md-3 text-truncate" style="line-height: 1.4">{{subTitle()}}</span>
       }
       @if (info()) {
-        <button class="btn btn-sm btn-link text-muted p-0 p-md-2" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
+        <button class="btn btn-sm btn-link text-muted p-0 ms-2 p-md-2 ms-md-0" title="What's this?" i18n-title type="button" [ngbPopover]="infoPopover" [autoClose]="true">
           <i-bs name="question-circle"></i-bs>
         </button>
         <ng-template #infoPopover>

src-ui/src/app/components/common/page-header/page-header.component.scss

@@ -1,5 +1,6 @@
 h3 {
     min-height: calc(1.325rem + 0.9vw);
+    flex-wrap: wrap;
 
     .badge {
         font-size: 0.65rem;
@@ -7,6 +8,16 @@ h3 {
     }
 }
 
+// Drop the subtitle onto its own and shrink it
+@media (max-width: 767.98px) {
+    h3 > .page-subtitle {
+        flex: 0 0 100%;
+        margin-top: 0 !important;
+        font-size: .7rem;
+        line-height: 1.3 !important;
+    }
+}
+
 :host {
   display: block;
   margin-bottom: .35rem;

src-ui/src/app/components/dashboard/widgets/welcome-widget/welcome-widget.component.html

@@ -1,10 +1,18 @@
-<ngb-alert class="pe-3" type="primary" [dismissible]="true" (closed)="dismiss.emit(true)">
-  <h4 class="alert-heading"><ng-container i18n>Paperless-ngx is running!</ng-container> 🎉</h4>
-  <p i18n>You're ready to start uploading documents! Explore the various features of this web app on your own, or start a quick tour using the button below.</p>
-  <p i18n>More detail on how to use and configure Paperless-ngx is always available in the <a href="https://docs.paperless-ngx.com" target="_blank">documentation</a>.</p>
-  <hr>
-  <div class="d-flex align-items-end">
-    <p class="lead fs-6 m-0"><em i18n>Thanks for being a part of the Paperless-ngx community!</em></p>
-    <button class="btn btn-primary ms-auto flex-shrink-0" (click)="tourService.start()"><ng-container i18n>Start the tour</ng-container> &rarr;</button>
+<ngb-alert class="welcome-widget bg-light shadow-sm pe-3 position-relative overflow-hidden" type="primary" [dismissible]="true" (closed)="dismiss.emit(true)">
+  <pngx-brand-mark class="welcome-watermark d-none d-md-block position-absolute top-50 translate-middle-y text-primary pe-none" width="26rem" height="26rem"></pngx-brand-mark>
+  <div class="d-flex align-items-start gap-3 gap-md-4 position-relative">
+    <pngx-brand-mark class="d-none d-sm-block flex-shrink-0 text-primary" width="3rem" height="3rem"></pngx-brand-mark>
+    <div class="flex-grow-1">
+      <h4 class="alert-heading text-primary fw-semibold"><ng-container i18n>Paperless-ngx is running!</ng-container> 🎉</h4>
+      <div class="welcome-copy text-dark">
+        <p i18n>You're ready to start uploading documents! Explore the various features of this web app on your own, or start a quick tour using the button below.</p>
+        <p class="mb-0" i18n>More detail on how to use and configure Paperless-ngx is always available in the <a href="https://docs.paperless-ngx.com" target="_blank">documentation</a>.</p>
+      </div>
+      <hr class="opacity-100">
+      <div class="d-flex flex-column flex-sm-row align-items-sm-center gap-3">
+        <p class="m-0 text-secondary"><em i18n>Thanks for being a part of the Paperless-ngx community!</em></p>
+        <button class="btn btn-primary ms-sm-auto flex-shrink-0 align-self-start align-self-sm-auto" (click)="tourService.start()"><ng-container i18n>Start the tour</ng-container> &rarr;</button>
+      </div>
+    </div>
   </div>
 </ngb-alert>

src-ui/src/app/components/dashboard/widgets/welcome-widget/welcome-widget.component.scss

@@ -0,0 +1,16 @@
+.welcome-widget {
+  border-color: color-mix(in srgb, var(--bs-primary) 50%, transparent);
+
+  hr {
+    border-top-color: color-mix(in srgb, var(--bs-primary) 25%, transparent);
+  }
+}
+
+.welcome-copy {
+  max-width: 68ch;
+}
+
+.welcome-watermark {
+  right: -6rem;
+  opacity: .07;
+}

src-ui/src/app/components/dashboard/widgets/welcome-widget/welcome-widget.component.ts

@@ -1,12 +1,13 @@
 import { Component, EventEmitter, Output, inject } from '@angular/core'
 import { NgbAlertModule } from '@ng-bootstrap/ng-bootstrap'
 import { TourService } from 'ngx-ui-tour-ng-bootstrap'
+import { BrandMarkComponent } from '../../../common/logo/brand-mark/brand-mark.component'
 
 @Component({
   selector: 'pngx-welcome-widget',
   templateUrl: './welcome-widget.component.html',
   styleUrls: ['./welcome-widget.component.scss'],
-  imports: [NgbAlertModule],
+  imports: [NgbAlertModule, BrandMarkComponent],
 })
 export class WelcomeWidgetComponent {
   readonly tourService = inject(TourService)

src-ui/src/app/components/document-detail/document-detail.component.spec.ts

@@ -963,12 +963,24 @@ describe('DocumentDetailComponent', () => {
     component.reprocess()
     const modalCloseSpy = jest.spyOn(openModal, 'close')
     openModal.componentInstance.confirmClicked.next()
-    expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] })
+    expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, false)
     expect(modalSpy).toHaveBeenCalled()
     expect(toastSpy).toHaveBeenCalled()
     expect(modalCloseSpy).toHaveBeenCalled()
   })
 
+  it('should pass remote OCR choice when reprocessing', () => {
+    initNormally()
+    const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')
+    reprocessSpy.mockReturnValue(of(true))
+    let openModal: NgbModalRef
+    modalService.activeInstances.subscribe((modal) => (openModal = modal[0]))
+    component.reprocess()
+    openModal.componentInstance.remoteOcr = true
+    openModal.componentInstance.confirmClicked.next()
+    expect(reprocessSpy).toHaveBeenCalledWith({ documents: [doc.id] }, true)
+  })
+
   it('should show error if redo ocr call fails', () => {
     initNormally()
     const reprocessSpy = jest.spyOn(documentService, 'reprocessDocuments')

src-ui/src/app/components/document-detail/document-detail.component.ts

@@ -97,6 +97,7 @@ import { ISODateAdapter } from 'src/app/utils/ngb-iso-date-adapter'
 import * as UTIF from 'utif'
 import { DocumentDetailFieldID } from '../admin/settings/settings.component'
 import { ConfirmDialogComponent } from '../common/confirm-dialog/confirm-dialog.component'
+import { ReprocessConfirmDialogComponent } from '../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
 import { PasswordRemovalConfirmDialogComponent } from '../common/confirm-dialog/password-removal-confirm-dialog/password-removal-confirm-dialog.component'
 import { CustomFieldsDropdownComponent } from '../common/custom-fields-dropdown/custom-fields-dropdown.component'
 import { CorrespondentEditDialogComponent } from '../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
@@ -1398,7 +1399,7 @@ export class DocumentDetailComponent
   }
 
   reprocess() {
-    let modal = this.modalService.open(ConfirmDialogComponent, {
+    let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
       backdrop: 'static',
     })
     modal.componentInstance.title = $localize`Reprocess confirm`
@@ -1409,7 +1410,10 @@ export class DocumentDetailComponent
     modal.componentInstance.confirmClicked.subscribe(() => {
       modal.componentInstance.buttonsEnabled.set(false)
       this.documentsService
-        .reprocessDocuments({ documents: [this.document().id] })
+        .reprocessDocuments(
+          { documents: [this.document().id] },
+          modal.componentInstance.remoteOcr
+        )
         .subscribe({
           next: () => {
             this.toastService.showInfo(

src-ui/src/app/components/document-list/bulk-editor/bulk-editor.component.spec.ts

@@ -1122,6 +1122,7 @@ describe('BulkEditorComponent', () => {
     req.flush(true)
     expect(req.request.body).toEqual({
       documents: [3, 4],
+      remote_ocr: false,
     })
     httpTestingController.match(
       `${environment.apiBaseUrl}documents/?page=1&page_size=50&ordering=-created&truncate_content=true&include_selection_data=true`

src-ui/src/app/components/document-list/bulk-editor/bulk-editor.component.ts

@@ -52,6 +52,7 @@ import { flattenTags } from 'src/app/utils/flatten-tags'
 import { queryParamsFromFilterRules } from 'src/app/utils/query-params'
 import { MergeAsVersionsConfirmDialogComponent } from '../../common/confirm-dialog/merge-as-versions-confirm-dialog/merge-as-versions-confirm-dialog.component'
 import { MergeConfirmDialogComponent } from '../../common/confirm-dialog/merge-confirm-dialog/merge-confirm-dialog.component'
+import { ReprocessConfirmDialogComponent } from '../../common/confirm-dialog/reprocess-confirm-dialog/reprocess-confirm-dialog.component'
 import { RotateConfirmDialogComponent } from '../../common/confirm-dialog/rotate-confirm-dialog/rotate-confirm-dialog.component'
 import { CorrespondentEditDialogComponent } from '../../common/edit-dialog/correspondent-edit-dialog/correspondent-edit-dialog.component'
 import { CustomFieldEditDialogComponent } from '../../common/edit-dialog/custom-field-edit-dialog/custom-field-edit-dialog.component'
@@ -916,7 +917,7 @@ export class BulkEditorComponent
   }
 
   reprocessSelected() {
-    let modal = this.modalService.open(ConfirmDialogComponent, {
+    let modal = this.modalService.open(ReprocessConfirmDialogComponent, {
       backdrop: 'static',
     })
     modal.componentInstance.title = $localize`Reprocess confirm`
@@ -930,7 +931,10 @@ export class BulkEditorComponent
         modal.componentInstance.buttonsEnabled.set(false)
         this.executeDocumentAction(
           modal,
-          this.documentService.reprocessDocuments(this.getSelectionQuery())
+          this.documentService.reprocessDocuments(
+            this.getSelectionQuery(),
+            modal.componentInstance.remoteOcr
+          )
         )
       })
   }

src-ui/src/app/data/paperless-config.ts

@@ -54,6 +54,10 @@ export const ConfigCategory = {
   AI: $localize`AI Settings`,
 }
 
+export const ConfigSection = {
+  RemoteOCR: $localize`Remote OCR`,
+}
+
 export const LLMEmbeddingBackendConfig = {
   OPENAI_LIKE: 'openai-like',
   HUGGINGFACE: 'huggingface',
@@ -65,6 +69,15 @@ export const LLMBackendConfig = {
   OLLAMA: 'ollama',
 }
 
+export const RemoteOCREngineConfig = {
+  AZURE_AI: 'azureai',
+}
+
+export const RemoteOCRModeConfig = {
+  ALWAYS: 'always',
+  WORKFLOW_ONLY: 'workflow_only',
+}
+
 export interface ConfigOption {
   key: string
   title: string
@@ -72,6 +85,7 @@ export interface ConfigOption {
   choices?: Array<{ id: string; name: string }>
   config_key?: string
   category: string
+  section?: string
   note?: string
 }
 
@@ -181,6 +195,43 @@ export const PaperlessConfigOptions: ConfigOption[] = [
     config_key: 'PAPERLESS_OCR_USER_ARGS',
     category: ConfigCategory.OCR,
   },
+  {
+    key: 'remote_ocr_engine',
+    title: $localize`Remote OCR Engine`,
+    type: ConfigOptionType.Select,
+    choices: mapToItems(RemoteOCREngineConfig),
+    config_key: 'PAPERLESS_REMOTE_OCR_ENGINE',
+    category: ConfigCategory.OCR,
+    section: ConfigSection.RemoteOCR,
+    note: $localize`Enabling remote OCR sends documents to a third-party service for processing. Consider the privacy implications as well as potential costs before enabling.`,
+  },
+  {
+    key: 'remote_ocr_api_key',
+    title: $localize`Remote OCR API Key`,
+    type: ConfigOptionType.Password,
+    config_key: 'PAPERLESS_REMOTE_OCR_API_KEY',
+    category: ConfigCategory.OCR,
+    section: ConfigSection.RemoteOCR,
+  },
+  {
+    key: 'remote_ocr_endpoint',
+    title: $localize`Remote OCR Endpoint`,
+    type: ConfigOptionType.String,
+    config_key: 'PAPERLESS_REMOTE_OCR_ENDPOINT',
+    category: ConfigCategory.OCR,
+    section: ConfigSection.RemoteOCR,
+    note: $localize`Required when using the Azure AI engine.`,
+  },
+  {
+    key: 'remote_ocr_mode',
+    title: $localize`Remote OCR Mode`,
+    type: ConfigOptionType.Select,
+    choices: mapToItems(RemoteOCRModeConfig),
+    config_key: 'PAPERLESS_REMOTE_OCR_MODE',
+    category: ConfigCategory.OCR,
+    section: ConfigSection.RemoteOCR,
+    note: $localize`Which documents are sent to the remote engine. Use 'workflow_only' to keep remote OCR off unless a workflow enables it for a document.`,
+  },
   {
     key: 'app_logo',
     title: $localize`Application Logo`,
@@ -398,6 +449,10 @@ export interface PaperlessConfig extends ObjectWithId {
   barcode_enable_tag: boolean
   barcode_tag_mapping: object
   barcode_tag_split: boolean
+  remote_ocr_engine: string
+  remote_ocr_api_key: string
+  remote_ocr_endpoint: string
+  remote_ocr_mode: string
   ai_enabled: boolean
   llm_embedding_backend: string
   llm_embedding_model: string

src-ui/src/app/data/ui-settings.ts

@@ -1,5 +1,6 @@
 import { PdfEditorEditMode } from '../components/common/pdf-editor/pdf-editor-edit-mode'
 import { PdfZoomScale } from '../components/common/pdf-viewer/pdf-viewer.types'
+import { RemoteOCRModeConfig } from './paperless-config'
 import { User } from './user'
 
 export interface UiSettings {
@@ -94,6 +95,8 @@ export const SETTINGS_KEYS = {
   OUTLOOK_OAUTH_URL: 'outlook_oauth_url',
   EMAIL_ENABLED: 'email_enabled',
   AI_ENABLED: 'ai_enabled',
+  REMOTE_OCR_CONFIGURED: 'remote_ocr:configured',
+  REMOTE_OCR_MODE: 'remote_ocr:mode',
 }
 
 export const SETTINGS: UiSetting[] = [
@@ -347,4 +350,14 @@ export const SETTINGS: UiSetting[] = [
     type: 'string',
     default: PdfEditorEditMode.Create,
   },
+  {
+    key: SETTINGS_KEYS.REMOTE_OCR_CONFIGURED,
+    type: 'boolean',
+    default: false,
+  },
+  {
+    key: SETTINGS_KEYS.REMOTE_OCR_MODE,
+    type: 'string',
+    default: RemoteOCRModeConfig.ALWAYS,
+  },
 ]

src-ui/src/app/data/workflow-action.ts

@@ -7,6 +7,18 @@ export enum WorkflowActionType {
   Webhook = 4,
   PasswordRemoval = 5,
   MoveToTrash = 6,
+  RemoteOcr = 7,
+  ApplyAiSuggestions = 8,
+}
+
+// see src/documents/models.py AISuggestionField
+export enum AISuggestionField {
+  Title = 'title',
+  Tags = 'tags',
+  Correspondent = 'correspondent',
+  DocumentType = 'document_type',
+  StoragePath = 'storage_path',
+  Created = 'created',
 }
 
 export interface WorkflowActionEmail extends ObjectWithId {
@@ -101,4 +113,10 @@ export interface WorkflowAction extends ObjectWithId {
   webhook?: WorkflowActionWebhook
 
   passwords?: string[]
+
+  ai_suggestion_fields?: AISuggestionField[]
+
+  ai_create_missing?: boolean
+
+  ai_overwrite_existing?: boolean
 }

src-ui/src/app/services/rest/document.service.spec.ts

@@ -284,6 +284,21 @@ describe(`DocumentService`, () => {
     expect(req.request.method).toEqual('POST')
     expect(req.request.body).toEqual({
       documents: ids,
+      remote_ocr: false,
+    })
+  })
+
+  it('should request remote OCR when reprocessing with it enabled', () => {
+    const ids = [1, 2, 3]
+    subscription = service
+      .reprocessDocuments({ documents: ids }, true)
+      .subscribe()
+    const req = httpTestingController.expectOne(
+      `${environment.apiBaseUrl}${endpoint}/reprocess/`
+    )
+    expect(req.request.body).toEqual({
+      documents: ids,
+      remote_ocr: true,
     })
   })
 

src-ui/src/app/services/rest/document.service.ts

@@ -349,9 +349,13 @@ export class DocumentService extends AbstractPaperlessService<Document> {
     })
   }
 
-  reprocessDocuments(selection: DocumentSelectionQuery) {
+  reprocessDocuments(
+    selection: DocumentSelectionQuery,
+    remoteOcr: boolean = false
+  ) {
     return this.http.post(this.getResourceUrl(null, 'reprocess'), {
       ...selection,
+      remote_ocr: remoteOcr,
     })
   }
 

src-ui/src/app/services/settings.service.spec.ts

@@ -13,6 +13,7 @@ import { environment } from 'src/environments/environment'
 import { CustomFieldDataType } from '../data/custom-field'
 import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
 import { SavedView } from '../data/saved-view'
+import { RemoteOCRModeConfig } from '../data/paperless-config'
 import { SETTINGS_KEYS, UiSettings } from '../data/ui-settings'
 import { PermissionsService } from './permissions.service'
 import { CustomFieldsService } from './rest/custom-fields.service'
@@ -434,4 +435,26 @@ describe('SettingsService', () => {
         ).name
     ).toEqual(customFields[0].name)
   })
+  it('should offer remote OCR only when configured and selective', () => {
+    settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, false)
+    settingsService.set(
+      SETTINGS_KEYS.REMOTE_OCR_MODE,
+      RemoteOCRModeConfig.WORKFLOW_ONLY
+    )
+    expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
+
+    // configured, but already handling every document
+    settingsService.set(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED, true)
+    settingsService.set(
+      SETTINGS_KEYS.REMOTE_OCR_MODE,
+      RemoteOCRModeConfig.ALWAYS
+    )
+    expect(settingsService.remoteOCRIsSelectable).toBeFalsy()
+
+    settingsService.set(
+      SETTINGS_KEYS.REMOTE_OCR_MODE,
+      RemoteOCRModeConfig.WORKFLOW_ONLY
+    )
+    expect(settingsService.remoteOCRIsSelectable).toBeTruthy()
+  })
 })

src-ui/src/app/services/settings.service.ts

@@ -19,6 +19,7 @@ import {
 } from 'src/app/utils/color'
 import { DEFAULT_APP_TITLE, environment } from 'src/environments/environment'
 import { DEFAULT_DISPLAY_FIELDS, DisplayField } from '../data/document'
+import { RemoteOCRModeConfig } from '../data/paperless-config'
 import { SavedView } from '../data/saved-view'
 import {
   PAPERLESS_GREEN_HEX,
@@ -687,6 +688,17 @@ export class SettingsService {
     return this.settingIsSet(SETTINGS_KEYS.UPDATE_CHECKING_ENABLED)
   }
 
+  /**
+   * Offering remote OCR as a  choice only makes sense when an engine
+   * is configured but is not already handling every document.
+   */
+  get remoteOCRIsSelectable(): boolean {
+    return (
+      this.get(SETTINGS_KEYS.REMOTE_OCR_CONFIGURED) &&
+      this.get(SETTINGS_KEYS.REMOTE_OCR_MODE) !== RemoteOCRModeConfig.ALWAYS
+    )
+  }
+
   offerTour(): boolean {
     return this.dashboardIsEmpty() && !this.get(SETTINGS_KEYS.TOUR_COMPLETE)
   }

src-ui/src/main.ts

@@ -118,6 +118,7 @@ import {
   journalBookmarkFill,
   journals,
   link,
+  list,
   listNested,
   listTask,
   listUl,
@@ -365,6 +366,7 @@ const icons = {
   journalBookmarkFill,
   journals,
   link,
+  list,
   listNested,
   listTask,
   listUl,

src-ui/src/styles.scss

@@ -536,6 +536,7 @@ ul.pagination {
 }
 
 .nav-underline {
+  gap: calc(0.5 * var(--bs-nav-underline-gap)) var(--bs-nav-underline-gap);
   .nav-link {
     &.active, &:hover, &:focus {
       color: var(--bs-primary);

src/documents/bulk_edit.py

@@ -399,10 +399,16 @@ def delete(doc_ids: list[int]) -> Literal["OK"]:
     return "OK"
 
 
-def reprocess(doc_ids: list[int]) -> Literal["OK"]:
+def reprocess(doc_ids: list[int], *, remote_ocr: bool = False) -> Literal["OK"]:
+    """
+    Re-run parsing for the given documents.
+
+    Consumption workflows do not run here, so ``remote_ocr`` is how the user
+    asks for the remote engine when it is not configured to handle everything.
+    """
     for document_id in doc_ids:
         update_document_content_maybe_archive_file.apply_async(
-            kwargs={"document_id": document_id},
+            kwargs={"document_id": document_id, "remote_ocr": remote_ocr},
             headers={"trigger_source": PaperlessTask.TriggerSource.MANUAL},
         )
 

src/documents/caching.py

@@ -2,12 +2,15 @@ from __future__ import annotations
 
 import logging
 import pickle
+import time
 from binascii import hexlify
 from collections import OrderedDict
 from dataclasses import dataclass
+from hashlib import sha256
 from typing import TYPE_CHECKING
 from typing import Any
 from typing import Final
+from uuid import uuid4
 
 from django.conf import settings
 from django.core.cache import cache
@@ -16,6 +19,7 @@ from django.core.cache import caches
 from documents.models import Document
 
 if TYPE_CHECKING:
+    from django.contrib.auth.models import User
     from django.core.cache.backends.base import BaseCache
 
     from documents.classifier import DocumentClassifier
@@ -52,6 +56,9 @@ CLASSIFIER_MODIFIED_KEY: Final[str] = "classifier_modified"
 #          [...]} per taxonomy field (#13676)
 LLM_CACHE_CLASSIFIER_VERSION: Final[int] = 1001
 
+# How often a request waiting on llm generation re-checks the cache
+LLM_SUGGESTION_POLL_INTERVAL: Final[float] = 0.5
+
 CACHE_1_MINUTE: Final[int] = 60
 CACHE_5_MINUTES: Final[int] = 5 * CACHE_1_MINUTE
 CACHE_50_MINUTES: Final[int] = 50 * CACHE_1_MINUTE
@@ -223,6 +230,68 @@ def get_llm_suggestion_cache(
     return None
 
 
+def retrieve_llm_suggestions(
+    document: Document,
+    user: User | None,
+    output_language: str | None,
+    *,
+    backend: str,
+    lock_timeout: int,
+) -> dict:
+    """Return cached LLM suggestions, generating them once across workers."""
+    # Lazy import to avoid pulling in the whole AI stuff
+    from paperless_ai.ai_classifier import get_ai_document_classification
+    from paperless_ai.exceptions import LLMTimeoutError
+
+    lock_key = (
+        f"{get_suggestion_cache_key(document.pk)}_llm_lock_"
+        f"{sha256(backend.encode()).hexdigest()}"
+    )
+    waited = False
+
+    while True:
+        cached = get_llm_suggestion_cache(document.pk, backend=backend)
+        if cached is not None:
+            refresh_suggestions_cache(document.pk)
+            return cached.suggestions
+
+        lock_token = uuid4().hex
+        if cache.add(lock_key, lock_token, lock_timeout):
+            if waited:
+                # The generation we were waiting on has ended without caching
+                # anything so it either failed or outlived its lock. Give up
+                # rather than re-running it
+                cache.delete(lock_key)
+                raise LLMTimeoutError
+
+            try:
+                # The cache may have been populated while acquiring the lock.
+                cached = get_llm_suggestion_cache(document.pk, backend=backend)
+                if cached is not None:
+                    refresh_suggestions_cache(document.pk)
+                    return cached.suggestions
+
+                suggestions = get_ai_document_classification(
+                    document,
+                    user,
+                    output_language,
+                )
+                set_llm_suggestions_cache(
+                    document.pk,
+                    suggestions,
+                    backend=backend,
+                )
+                return suggestions
+            finally:
+                # Don't remove lock if this one expired while generation was still running
+                if cache.get(lock_key) == lock_token:
+                    cache.delete(lock_key)
+
+        waited = True
+        # Another worker is generating suggestions, poll to avoid another LLM request
+        time.sleep(LLM_SUGGESTION_POLL_INTERVAL)
+
+
 def set_llm_suggestions_cache(
     document_id: int,
     suggestions: dict,

src/documents/consumer.py

@@ -53,6 +53,7 @@ from documents.utils import copy_basic_file_stats
 from documents.utils import copy_file_with_basic_stats
 from documents.utils import run_subprocess
 from paperless.config import OcrConfig
+from paperless.config import RemoteOCRConfig
 from paperless.models import ArchiveFileGenerationChoices
 from paperless.parsers import ParserContext
 from paperless.parsers import ParserProtocol
@@ -451,12 +452,19 @@ class ConsumerPlugin(
                 except Exception as e:
                     self.log.error(f"Error attempting to clean PDF: {e}")
 
+            # Workflows have already run at this point, so the metadata knows
+            # whether this document was singled out for remote OCR
+            allow_remote = (
+                self.metadata.remote_ocr or RemoteOCRConfig().remote_ocr_by_default
+            )
+
             # Based on the mime type, get the parser for that type
             parser_class: type[ParserProtocol] | None = (
                 get_parser_registry().get_parser_for_file(
                     mime_type,
                     self.filename,
                     self.working_copy,
+                    allow_remote=allow_remote,
                 )
             )
             if not parser_class:
@@ -465,6 +473,16 @@ class ConsumerPlugin(
                     f"Unsupported mime type {mime_type}",
                 )
 
+            if self.metadata.remote_ocr and not getattr(
+                parser_class,
+                "uses_remote_service",
+                False,
+            ):
+                self.log.warning(
+                    "Remote OCR was requested for this document but no remote "
+                    "parser is available for it, processing locally instead.",
+                )
+
             # Notify all listeners that we're going to do some work.
 
             document_consumption_started.send(

src/documents/data_models.py

@@ -34,6 +34,7 @@ class DocumentMetadataOverrides:
     skip_asn_if_exists: bool = False
     version_label: str | None = None
     actor_id: int | None = None
+    remote_ocr: bool = False
 
     def update(self, other: "DocumentMetadataOverrides") -> "DocumentMetadataOverrides":
         """
@@ -57,6 +58,8 @@ class DocumentMetadataOverrides:
             self.actor_id = other.actor_id
         if other.skip_asn_if_exists:
             self.skip_asn_if_exists = True
+        if other.remote_ocr:
+            self.remote_ocr = True
         if other.version_label is not None:
             self.version_label = other.version_label
 

src/documents/migrations/0024_alter_workflowaction_type.py

@@ -0,0 +1,30 @@
+# Generated by Django 5.2.16 on 2026-08-10 17:27
+
+from django.db import migrations
+from django.db import models
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("documents", "0023_savedview_icon"),
+    ]
+
+    operations = [
+        migrations.AlterField(
+            model_name="workflowaction",
+            name="type",
+            field=models.PositiveSmallIntegerField(
+                choices=[
+                    (1, "Assignment"),
+                    (2, "Removal"),
+                    (3, "Email"),
+                    (4, "Webhook"),
+                    (5, "Password removal"),
+                    (6, "Move to trash"),
+                    (7, "Remote OCR"),
+                ],
+                default=1,
+                verbose_name="Workflow Action Type",
+            ),
+        ),
+    ]

src/documents/migrations/0025_workflowaction_apply_ai_suggestions.py

@@ -0,0 +1,84 @@
+# Generated by Django 5.2.16 on 2026-08-10 18:26
+
+from django.db import migrations
+from django.db import models
+
+
+class Migration(migrations.Migration):
+    dependencies = [
+        ("documents", "0024_alter_workflowaction_type"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="workflowaction",
+            name="ai_create_missing",
+            field=models.BooleanField(
+                default=False,
+                help_text="Create suggested tags, correspondents, document types and storage paths that do not already exist instead of skipping them.",
+                verbose_name="create missing objects",
+            ),
+        ),
+        migrations.AddField(
+            model_name="workflowaction",
+            name="ai_overwrite_existing",
+            field=models.BooleanField(
+                default=False,
+                help_text="Apply suggestions even if the document already has a value for that field. Tags are always added to, never replaced.",
+                verbose_name="overwrite existing values",
+            ),
+        ),
+        migrations.AddField(
+            model_name="workflowaction",
+            name="ai_suggestion_fields",
+            field=models.JSONField(
+                blank=True,
+                help_text="Which of the AI-suggested fields to apply to the document.",
+                null=True,
+                verbose_name="AI suggestion fields",
+            ),
+        ),
+        migrations.AlterField(
+            model_name="workflowaction",
+            name="type",
+            field=models.PositiveSmallIntegerField(
+                choices=[
+                    (1, "Assignment"),
+                    (2, "Removal"),
+                    (3, "Email"),
+                    (4, "Webhook"),
+                    (5, "Password removal"),
+                    (6, "Move to trash"),
+                    (7, "Remote OCR"),
+                    (8, "Apply AI suggestions"),
+                ],
+                default=1,
+                verbose_name="Workflow Action Type",
+            ),
+        ),
+        migrations.AlterField(
+            model_name="paperlesstask",
+            name="task_type",
+            field=models.CharField(
+                choices=[
+                    ("consume_file", "Consume File"),
+                    ("train_classifier", "Train Classifier"),
+                    ("sanity_check", "Sanity Check"),
+                    ("index_optimize", "Index Optimize"),
+                    ("mail_fetch", "Mail Fetch"),
+                    ("llm_index", "LLM Index"),
+                    ("empty_trash", "Empty Trash"),
+                    ("check_workflows", "Check Workflows"),
+                    ("bulk_update", "Bulk Update"),
+                    ("reprocess_document", "Reprocess Document"),
+                    ("build_share_link", "Build Share Link"),
+                    ("bulk_delete", "Bulk Delete"),
+                    ("apply_ai_suggestions", "Apply AI Suggestions"),
+                ],
+                db_index=True,
+                help_text="The kind of work being performed",
+                max_length=50,
+                verbose_name="Task Type",
+            ),
+        ),
+    ]

src/documents/models.py

@@ -766,6 +766,7 @@ class PaperlessTask(ModelWithOwner):
         REPROCESS_DOCUMENT = "reprocess_document", _("Reprocess Document")
         BUILD_SHARE_LINK = "build_share_link", _("Build Share Link")
         BULK_DELETE = "bulk_delete", _("Bulk Delete")
+        APPLY_AI_SUGGESTIONS = "apply_ai_suggestions", _("Apply AI Suggestions")
 
     COMPLETE_STATUSES = (
         Status.SUCCESS,
@@ -1670,6 +1671,22 @@ class WorkflowAction(models.Model):
             6,
             _("Move to trash"),
         )
+        REMOTE_OCR = (
+            7,
+            _("Remote OCR"),
+        )
+        APPLY_AI_SUGGESTIONS = (
+            8,
+            _("Apply AI suggestions"),
+        )
+
+    class AISuggestionField(models.TextChoices):
+        TITLE = ("title", _("Title"))
+        TAGS = ("tags", _("Tags"))
+        CORRESPONDENT = ("correspondent", _("Correspondent"))
+        DOCUMENT_TYPE = ("document_type", _("Document type"))
+        STORAGE_PATH = ("storage_path", _("Storage path"))
+        CREATED = ("created", _("Created date"))
 
     type = models.PositiveSmallIntegerField(
         _("Workflow Action Type"),
@@ -1908,6 +1925,33 @@ class WorkflowAction(models.Model):
         ),
     )
 
+    ai_suggestion_fields = models.JSONField(
+        _("AI suggestion fields"),
+        null=True,
+        blank=True,
+        help_text=_(
+            "Which of the AI-suggested fields to apply to the document.",
+        ),
+    )
+
+    ai_create_missing = models.BooleanField(
+        _("create missing objects"),
+        default=False,
+        help_text=_(
+            "Create suggested tags, correspondents, document types and storage "
+            "paths that do not already exist instead of skipping them.",
+        ),
+    )
+
+    ai_overwrite_existing = models.BooleanField(
+        _("overwrite existing values"),
+        default=False,
+        help_text=_(
+            "Apply suggestions even if the document already has a value for that "
+            "field. Tags are always added to, never replaced.",
+        ),
+    )
+
     class Meta:
         verbose_name = _("workflow action")
         verbose_name_plural = _("workflow actions")

src/documents/search/__init__.py

@@ -6,20 +6,13 @@ from documents.search._backend import TantivyRelevanceList
 from documents.search._backend import WriteBatch
 from documents.search._backend import get_backend
 from documents.search._backend import reset_backend
-from documents.search._errors import InvalidDateQuery
-from documents.search._errors import InvalidNumberQuery
-from documents.search._errors import MultipleSearchQueryErrors
-from documents.search._errors import QueryTooLongError
-from documents.search._errors import SearchQueryError
-from documents.search._errors import search_query_error_messages
 from documents.search._schema import needs_rebuild
 from documents.search._schema import wipe_index
+from documents.search._translate import InvalidDateQuery
+from documents.search._translate import SearchQueryError
 
 __all__ = [
     "InvalidDateQuery",
-    "InvalidNumberQuery",
-    "MultipleSearchQueryErrors",
-    "QueryTooLongError",
     "SearchHit",
     "SearchIndexLockError",
     "SearchMode",
@@ -30,6 +23,5 @@ __all__ = [
     "get_backend",
     "needs_rebuild",
     "reset_backend",
-    "search_query_error_messages",
     "wipe_index",
 ]

src/documents/search/_backend.py

@@ -22,6 +22,7 @@ import tantivy
 from django.conf import settings
 from django.utils.timezone import get_current_timezone
 
+from documents.search._query import build_permission_filter
 from documents.search._query import extract_cjk_text
 from documents.search._query import parse_simple_text_highlight_query
 from documents.search._query import parse_simple_text_query
@@ -39,7 +40,6 @@ from documents.utils import QuerySetStream
 from documents.utils import identity
 
 if TYPE_CHECKING:
-    from collections.abc import Iterable
     from collections.abc import Iterator
     from collections.abc import Sequence
     from pathlib import Path
@@ -285,47 +285,6 @@ class WriteBatch:
         )
 
 
-def build_permission_filter(
-    schema: tantivy.Schema,
-    user: AbstractUser,
-    viewer_group_ids: Iterable[int] = (),
-) -> tantivy.Query:
-    """
-    Build a query filter for user document permissions.
-
-    Creates a query that matches only documents visible to the specified user
-    according to paperless-ngx permission rules:
-    - Public documents (no owner) are visible to all users
-    - Private documents are visible to their owner
-    - Documents explicitly shared with the user are visible
-    - Documents shared with one of the user's current groups are visible
-
-    Args:
-        schema: Tantivy schema for field validation
-        user: User to check permissions for
-        viewer_group_ids: Current group memberships for the user
-
-    Returns:
-        Tantivy query that filters results to visible documents
-    """
-    owner_any = tantivy.Query.exists_query("owner_id")
-    no_owner = tantivy.Query.boolean_query(
-        [
-            (tantivy.Occur.Must, tantivy.Query.all_query()),
-            (tantivy.Occur.MustNot, owner_any),
-        ],
-    )
-    owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
-    shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
-    group_shared = [
-        tantivy.Query.term_query(schema, "viewer_group_id", group_id)
-        for group_id in viewer_group_ids
-    ]
-    return tantivy.Query.disjunction_max_query(
-        [no_owner, owned, shared, *group_shared],
-    )
-
-
 class TantivyBackend:
     """
     Tantivy search backend with explicit lifecycle management.
@@ -499,6 +458,7 @@ class TantivyBackend:
             doc.add_text("correspondent_sort", document.correspondent.name)
             if cjk_corr := extract_cjk_text(document.correspondent.name):
                 doc.add_text("bigram_correspondent", cjk_corr)
+            doc.add_unsigned("correspondent_id", document.correspondent_id)
 
         # Document type
         if document.document_type:
@@ -506,10 +466,12 @@ class TantivyBackend:
             doc.add_text("type_sort", document.document_type.name)
             if cjk_type := extract_cjk_text(document.document_type.name):
                 doc.add_text("bigram_document_type", cjk_type)
+            doc.add_unsigned("document_type_id", document.document_type_id)
 
         # Storage path
         if document.storage_path:
             doc.add_text("storage_path", document.storage_path.name)
+            doc.add_unsigned("storage_path_id", document.storage_path_id)
 
         # Tags — collect names for autocomplete in the same pass
         tag_names: list[str] = []
@@ -517,13 +479,12 @@ class TantivyBackend:
             doc.add_text("tag", tag.name)
             if cjk_tag := extract_cjk_text(tag.name):
                 doc.add_text("bigram_tag", cjk_tag)
+            doc.add_unsigned("tag_id", tag.pk)
             tag_names.append(tag.name)
 
         # Notes — JSON for structured queries (notes.user:alice, notes.note:text).
         # notes_text is a plain-text companion for snippet/highlight generation;
-        # tantivy's SnippetGenerator does not support JSON fields. It is not in
-        # _DEFAULT_SEARCH_FIELDS, so an unqualified query never searches it: a
-        # note matches through the JSON field or not at all.
+        # tantivy's SnippetGenerator does not support JSON fields.
         num_notes = 0
         note_texts: list[str] = []
         for note in document.notes.all():
@@ -539,9 +500,8 @@ class TantivyBackend:
         if note_texts:
             doc.add_text("notes_text", " ".join(note_texts))
 
-        # Custom fields: JSON for structured queries (custom_fields.name:x,
-        # custom_fields.value:y). There is no companion text field here, unlike
-        # notes: custom field values are reachable only through the JSON field.
+        # Custom fields — JSON for structured queries (custom_fields.name:x, custom_fields.value:y),
+        # companion text field for default full-text search.
         for cfi in document.custom_fields.all():
             search_value = cfi.value_for_search
             # Skip fields where there is no value yet
@@ -708,17 +668,7 @@ class TantivyBackend:
         user_query = self._parse_query(query, search_mode)
         highlight_query = user_query
         if search_mode is SearchMode.TEXT:
-            try:
-                highlight_query = parse_simple_text_highlight_query(
-                    self._index,
-                    query,
-                )
-            except ValueError:
-                logger.debug(
-                    "Skipping simple text highlight query: token string is not "
-                    "valid tantivy query syntax: %r",
-                    query,
-                )
+            highlight_query = parse_simple_text_highlight_query(self._index, query)
 
         # For notes_text snippet generation, we need a query that targets the
         # notes_text field directly. user_query may contain JSON-field terms

src/documents/search/_dates.py

@@ -0,0 +1,171 @@
+from __future__ import annotations
+
+from datetime import UTC
+from datetime import date
+from datetime import datetime
+from datetime import timedelta
+from typing import TYPE_CHECKING
+from typing import Final
+
+from dateutil.relativedelta import relativedelta
+
+if TYPE_CHECKING:
+    from datetime import tzinfo
+
+_DATE_ONLY_FIELDS = frozenset({"created"})
+
+_TODAY: Final[str] = "today"
+_YESTERDAY: Final[str] = "yesterday"
+_PREVIOUS_WEEK: Final[str] = "previous week"
+_THIS_MONTH: Final[str] = "this month"
+_PREVIOUS_MONTH: Final[str] = "previous month"
+_THIS_YEAR: Final[str] = "this year"
+_PREVIOUS_YEAR: Final[str] = "previous year"
+_PREVIOUS_QUARTER: Final[str] = "previous quarter"
+
+_DATE_KEYWORDS = frozenset(
+    {
+        _TODAY,
+        _YESTERDAY,
+        _PREVIOUS_WEEK,
+        _THIS_MONTH,
+        _PREVIOUS_MONTH,
+        _THIS_YEAR,
+        _PREVIOUS_YEAR,
+        _PREVIOUS_QUARTER,
+    },
+)
+
+
+def _fmt(dt: datetime) -> str:
+    """Format a datetime as an ISO 8601 UTC string for use in Tantivy range queries."""
+    return dt.astimezone(UTC).strftime("%Y-%m-%dT%H:%M:%SZ")
+
+
+def _iso_range(lo: datetime, hi: datetime) -> str:
+    """
+    Format a half-open ``[lo TO hi)`` range in ISO 8601 for Tantivy query syntax.
+
+    ``hi`` is always the exclusive ceiling of a computed period (the start of
+    the *next* day/week/month/quarter/year), so the closing bracket must be
+    the Tantivy exclusive-range brace ``}`` rather than ``]`` — otherwise the
+    first instant of the following period (e.g. the 1st of next month) is
+    incorrectly included in the match.
+    """
+    return f"[{_fmt(lo)} TO {_fmt(hi)}}}"
+
+
+def _quarter_start(d: date) -> date:
+    """Return the first day of the calendar quarter containing ``d``."""
+    return date(d.year, ((d.month - 1) // 3) * 3 + 1, 1)
+
+
+def _midnight(d: date, tz: tzinfo) -> datetime:
+    """Convert a calendar date at local-timezone midnight to a UTC datetime."""
+    return datetime(d.year, d.month, d.day, tzinfo=tz).astimezone(UTC)
+
+
+def _keyword_bounds(keyword: str, tz: tzinfo) -> tuple[date, date]:
+    """
+    Map a relative date keyword to ``(start, exclusive_end)`` calendar dates.
+
+    ``tz`` only determines what "today" is; the caller decides how the returned
+    dates become UTC datetime boundaries (date-only vs. local-midnight offset).
+    """
+    today = datetime.now(tz).date()
+    if keyword == _TODAY:
+        return today, today + timedelta(days=1)
+    if keyword == _YESTERDAY:
+        return today - timedelta(days=1), today
+    if keyword == _PREVIOUS_WEEK:
+        this_monday = today - timedelta(days=today.weekday())
+        return this_monday - timedelta(weeks=1), this_monday
+    if keyword == _THIS_MONTH:
+        first = today.replace(day=1)
+        return first, first + relativedelta(months=1)
+    if keyword == _PREVIOUS_MONTH:
+        this_first = today.replace(day=1)
+        return this_first - relativedelta(months=1), this_first
+    if keyword == _THIS_YEAR:
+        return date(today.year, 1, 1), date(today.year + 1, 1, 1)
+    if keyword == _PREVIOUS_YEAR:
+        return date(today.year - 1, 1, 1), date(today.year, 1, 1)
+    if keyword == _PREVIOUS_QUARTER:
+        this_quarter = _quarter_start(today)
+        return this_quarter - relativedelta(months=3), this_quarter
+    raise ValueError(f"Unknown keyword: {keyword}")
+
+
+def _date_only_range(keyword: str, tz: tzinfo) -> str:
+    """
+    For `created` (DateField): use the local calendar date, converted to
+    midnight UTC boundaries. No offset arithmetic — date only.
+    """
+    start, end = _keyword_bounds(keyword, tz)
+    lo = datetime(start.year, start.month, start.day, tzinfo=UTC)
+    hi = datetime(end.year, end.month, end.day, tzinfo=UTC)
+    return _iso_range(lo, hi)
+
+
+def _datetime_range(keyword: str, tz: tzinfo) -> str:
+    """
+    For `added` / `modified` (DateTimeField, stored as UTC): convert local day
+    boundaries to UTC — full offset arithmetic required.
+    """
+    start, end = _keyword_bounds(keyword, tz)
+    return _iso_range(_midnight(start, tz), _midnight(end, tz))
+
+
+def _precision_bounds(digits: str) -> tuple[date, date] | None:
+    """
+    Map a 4/6/8-digit date token to (start, exclusive_end) calendar dates.
+
+    YYYY -> whole year, YYYYMM -> whole month, YYYYMMDD -> single day.
+    Returns None for any unparsable or out-of-range value (e.g. month 23),
+    so callers can emit a no-match clause instead of erroring (Whoosh parity).
+    """
+    try:
+        if len(digits) == 4:
+            year = int(digits)
+            return date(year, 1, 1), date(year + 1, 1, 1)
+        if len(digits) == 6:
+            year, month = int(digits[:4]), int(digits[4:6])
+            start = date(year, month, 1)
+            end = date(year + 1, 1, 1) if month == 12 else date(year, month + 1, 1)
+            return start, end
+        if len(digits) == 8:
+            start = date(int(digits[:4]), int(digits[4:6]), int(digits[6:8]))
+            return start, start + timedelta(days=1)
+    except ValueError:
+        return None
+    return None
+
+
+def _utc_bounds_for_field(
+    field: str,
+    start: date,
+    end: date,
+    tz: tzinfo,
+) -> tuple[datetime, datetime]:
+    """
+    Convert calendar-date bounds to UTC datetimes per the field's storage type.
+
+    For DateField (``created``) the bounds are UTC midnight (no offset). For
+    DateTimeField (``added``/``modified``) the bounds are local-tz midnight
+    converted to UTC, matching how each field is indexed.
+    """
+    if field in _DATE_ONLY_FIELDS:
+        return (
+            datetime(start.year, start.month, start.day, tzinfo=UTC),
+            datetime(end.year, end.month, end.day, tzinfo=UTC),
+        )
+    return (
+        datetime(start.year, start.month, start.day, tzinfo=tz).astimezone(UTC),
+        datetime(end.year, end.month, end.day, tzinfo=tz).astimezone(UTC),
+    )
+
+
+def _field_range_from_dates(field: str, start: date, end: date, tz: tzinfo) -> str:
+    """Build a Tantivy ``field:[lo TO hi]`` ISO range from calendar-date bounds."""
+    lo, hi = _utc_bounds_for_field(field, start, end, tz)
+    return f"{field}:{_iso_range(lo, hi)}"

src/documents/search/_errors.py

@@ -1,71 +0,0 @@
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-if TYPE_CHECKING:
-    from collections.abc import Sequence
-
-
-class SearchQueryError(ValueError):
-    """
-    Base for user-fixable search query errors.
-
-    Carries a message safe to surface to the user (no internal details). The
-    view layer catches this and returns an HTTP 400, so any future subclass
-    gets the same treatment.
-    """
-
-
-class InvalidDateQuery(SearchQueryError):
-    """Raised when a date field value or range bound cannot be parsed."""
-
-    def __init__(self, field: str | None, value: str | None) -> None:
-        self.field = field
-        self.value = value
-        super().__init__(f"Invalid date value {value!r} for field {field!r}.")
-
-
-class InvalidNumberQuery(SearchQueryError):
-    """Raised when a numeric field value or range bound cannot be parsed."""
-
-    def __init__(self, field: str | None, value: str | None) -> None:
-        self.field = field
-        self.value = value
-        super().__init__(f"Invalid numeric value {value!r} for field {field!r}.")
-
-
-class QueryTooLongError(SearchQueryError):
-    """Raised when a query string exceeds the maximum allowed length.
-
-    whoosh-compat's fieldname tagger is O(n^2) in plain word characters, so an
-    unbounded query is a CPU-exhaustion vector against a single request
-    handler. This is a hard boundary, not a validation nicety.
-    """
-
-    def __init__(self, length: int, limit: int) -> None:
-        self.length = length
-        self.limit = limit
-        super().__init__(
-            f"The search query is too long ({length} characters). "
-            f"The maximum allowed length is {limit} characters.",
-        )
-
-
-class MultipleSearchQueryErrors(SearchQueryError):
-    """Aggregates every user-fixable error from one parse, not just the first."""
-
-    def __init__(self, errors: Sequence[SearchQueryError]) -> None:
-        self.errors = tuple(errors)
-        super().__init__("; ".join(str(e) for e in self.errors))
-
-
-def search_query_error_messages(e: SearchQueryError) -> list[str]:
-    """The user-facing message list for a SearchQueryError.
-
-    Every offending value's message, not just the first, so the user can
-    fix them all in one round-trip. Shared by every view that maps
-    SearchQueryError to an HTTP 400.
-    """
-    if isinstance(e, MultipleSearchQueryErrors):
-        return [str(sub) for sub in e.errors]
-    return [str(e)]

src/documents/search/_fields.py

@@ -1,42 +0,0 @@
-from __future__ import annotations
-
-from whoosh_compat import FieldKind
-from whoosh_compat import FieldSpec
-from whoosh_compat import SubpathSpec
-
-# Internal-only schema fields with no query-syntax meaning of their own
-# (sort shadow fields, bigram CJK fields, simple_title/simple_content,
-# autocomplete_word, notes_text) are NOT represented here, they are
-# declared in _schema.py's field_descriptors().
-#
-# analyzer/pattern_normalizer are deliberately left at FieldSpec's default
-# (None): they're language-specific and only meaningful to whoosh-compat's
-# parser, so _registry.py attaches them per-language via dataclasses.replace()
-# rather than PUBLIC_FIELDS declaring them itself. _schema.py only reads
-# name/kind/fast and never sees the analyzer at all.
-PUBLIC_FIELDS: tuple[FieldSpec, ...] = (
-    FieldSpec("title", FieldKind.TEXT),
-    FieldSpec("content", FieldKind.TEXT),
-    FieldSpec("correspondent", FieldKind.TEXT),
-    FieldSpec("document_type", FieldKind.TEXT, aliases=("type",)),
-    FieldSpec("storage_path", FieldKind.TEXT, aliases=("path",)),
-    FieldSpec("original_filename", FieldKind.TEXT),
-    FieldSpec("tag", FieldKind.TEXT, comma_values=True),
-    FieldSpec("checksum", FieldKind.KEYWORD),
-    FieldSpec("asn", FieldKind.U64, fast=True),
-    FieldSpec("page_count", FieldKind.U64, fast=True),
-    FieldSpec("num_notes", FieldKind.U64, fast=True),
-    FieldSpec("created", FieldKind.DATE, date_only=True, fast=True),
-    FieldSpec("modified", FieldKind.DATETIME, fast=True),
-    FieldSpec("added", FieldKind.DATETIME, fast=True),
-    FieldSpec(
-        "notes",
-        FieldKind.JSON,
-        subpaths={"user": SubpathSpec(), "note": SubpathSpec(default=True)},
-    ),
-    FieldSpec(
-        "custom_fields",
-        FieldKind.JSON,
-        subpaths={"name": SubpathSpec(), "value": SubpathSpec(default=True)},
-    ),
-)

src/documents/search/_query.py

@@ -6,30 +6,22 @@ from typing import Final
 
 import regex
 import tantivy
-import whoosh_compat as wc
 from django.conf import settings
-from whoosh_compat.emitters.tantivy_ import emit as tantivy_emit
-from whoosh_compat.errors import Cause
-from whoosh_compat.errors import Diagnostic
-from whoosh_compat.errors import DiagnosticKind
-from whoosh_compat.errors import QueryError
 
-from documents.search._errors import InvalidDateQuery
-from documents.search._errors import InvalidNumberQuery
-from documents.search._errors import MultipleSearchQueryErrors
-from documents.search._errors import SearchQueryError
-from documents.search._registry import get_field_registry
 from documents.search._tokenizer import simple_search_tokens
+from documents.search._translate import SearchQueryError
+from documents.search._translate import translate_query
 
 if TYPE_CHECKING:
+    from collections.abc import Iterable
     from datetime import tzinfo
 
+    from django.contrib.auth.base_user import AbstractBaseUser
+
 logger = logging.getLogger("paperless.search")
 
-# Maximum seconds any single regex substitution over user-supplied query text
-# may run. The one remaining use is a character class, which cannot backtrack,
-# so the bound is an upper limit on that substitution's cost, not the ReDoS
-# guard it was originally written as.
+# Maximum seconds any single regex substitution may run.
+# Prevents ReDoS on adversarial user-supplied query strings.
 _REGEX_TIMEOUT: Final[float] = 1.0
 
 # Matches CJK/Hangul characters so queries can be routed to bigram fields.
@@ -37,64 +29,6 @@ _REGEX_TIMEOUT: Final[float] = 1.0
 _CJK_RE: Final = regex.compile(r"[\p{Han}\p{Hiragana}\p{Katakana}\p{Hangul}]+")
 
 
-def _user_facing_emit_message(d: Diagnostic) -> str:
-    """A user-safe message for an emit-time QueryError's Diagnostic.
-
-    Built from the Diagnostic's structured fields (kind, field), never from
-    d.message: whoosh-compat documents that as developer/log output with no
-    stability guarantee, and PATTERN_TOO_COMPLEX embeds the raw backend
-    error text in it.
-    """
-    field = str(d.field) if d.field is not None else None
-    if d.kind is DiagnosticKind.EXISTS_REQUIRES_FAST:
-        return f"Existence searches (field:*) are not supported for field {field!r}."
-    if d.kind is DiagnosticKind.TEXT_RANGE:
-        return f"Range searches are not supported for field {field!r}."
-    if d.kind is DiagnosticKind.PATTERN_TOO_COMPLEX:
-        return f"The wildcard pattern for field {field!r} is too complex."
-    if d.kind is DiagnosticKind.SCHEMA_FIELD_MISSING:
-        return f"Field {field!r} is not available in the search index."
-    logger.warning("Unmapped emit diagnostic %s: %s", d.kind, d.message)
-    return "The search query could not be executed."
-
-
-def _map_emit_error(e: QueryError) -> SearchQueryError:
-    """Route an emit-time QueryError by its Diagnostic's Cause.
-
-    INVALID_INPUT/UNSUPPORTED are user-input errors, exactly like a parse
-    diagnostic, and map to a 400. INTERNAL means a defect in whoosh-compat
-    or in our own AST handling, never the user's query, so the QueryError is
-    re-raised rather than converted, reaching the generic 500 handler instead
-    of blaming the query. MISCONFIGURED is deliberately both: the registry and
-    the index schema disagree, which only an operator can fix, so it is logged
-    as an error, but a request is still waiting and the query cannot run
-    either way, so it also returns a 400.
-
-    EXISTS_REQUIRES_FAST is the one MISCONFIGURED kind that is not a
-    disagreement. whoosh-compat derives it from the registry's own FieldSpec
-    (kind plus fast) without ever consulting the index schema, so it fires
-    whenever a non-fast field of a kind that cannot answer "exists" is asked
-    to: for us that is only the JSON fields, which field_descriptors() builds
-    non-fast on purpose. "notes:*" and the five other spellings of it are
-    ordinary user error that no operator action can clear, so they get the
-    400 without the alert.
-    """
-    d = e.diagnostic
-    if d.cause is Cause.INTERNAL:
-        raise e
-    if (
-        d.cause is Cause.MISCONFIGURED
-        and d.kind is not DiagnosticKind.EXISTS_REQUIRES_FAST
-    ):
-        logger.error(
-            "Search index misconfiguration for field %s (%s): %s",
-            d.field,
-            d.kind.name,
-            d.message,
-        )
-    return SearchQueryError(_user_facing_emit_message(d))
-
-
 def _has_cjk(text: str) -> bool:
     """Return True if text contains any CJK characters."""
     return bool(_CJK_RE.search(text))
@@ -103,36 +37,14 @@ def _has_cjk(text: str) -> bool:
 def extract_cjk_text(text: str) -> str:
     """Join the CJK runs in ``text`` for indexing into bigram (char-ngram) fields.
 
-    Mirrors the query side, which extracts the CJK runs of whatever it is
-    about to search for (the raw string in simple modes, the parsed query's
-    free-text tokens in query mode): only CJK runs are ever searched against
-    the bigram fields, so only CJK runs are worth indexing there. Latin text
-    fed to a character-bigram field is never matched and only bloats the
+    Mirrors the query side (``_build_cjk_query``): only CJK runs are ever searched
+    against the bigram fields, so only CJK runs are worth indexing there. Latin
+    text fed to a character-bigram field is never matched and only bloats the
     index and slows indexing/merge. Returns "" when there is no CJK text.
     """
     return " ".join(_CJK_RE.findall(text))
 
 
-def _parse_cjk_text(
-    index: tantivy.Index,
-    cjk_text: str,
-    fields: list[str],
-) -> tantivy.Query | None:
-    """Parse a plain CJK run string against ``fields``, or None if it won't parse."""
-    try:
-        return index.parse_query(cjk_text, fields)
-    except Exception:
-        # Broad on purpose, unlike _try_parse_fuzzy_query's narrower
-        # ValueError: cjk_text isn't filtered to a guaranteed-safe token
-        # set the way the fuzzy blend's word string is, so the exact
-        # failure mode tantivy could raise here isn't pinned down.
-        logger.debug(
-            "Skipping CJK search clause: could not parse CJK text: %r",
-            cjk_text,
-        )
-        return None
-
-
 def _build_cjk_query(
     index: tantivy.Index,
     raw_query: str,
@@ -140,259 +52,91 @@ def _build_cjk_query(
 ) -> tantivy.Query | None:
     """Build a bigram-field query from the CJK runs in ``raw_query``.
 
-    For the simple (TEXT/TITLE) modes, whose input is plain text and carries
-    no query grammar to respect. Only the CJK character runs are extracted, so
-    a stray ``field:`` prefix or ``-``/``+`` in the input can neither leak
-    field semantics nor fail the parse, and no Latin token reaches the
-    character-bigram matcher (where it would produce spurious matches against
-    unrelated Latin text). Returns None when there is no CJK text or the parse
-    fails.
+    Only the CJK character runs are extracted and parsed; ASCII field prefixes,
+    boolean operators and date keywords are discarded. This keeps the CJK clause
+    plain-text and consistent across query/simple modes (no leaked ``field:``
+    semantics, no parse failures from spaced ``-``/``+``), and avoids feeding
+    Latin tokens into the character-bigram matcher (which would produce spurious
+    matches against unrelated Latin text). Returns None when there is no CJK
+    text or the parse fails.
     """
-    cjk_text = extract_cjk_text(raw_query)
+    cjk_text = " ".join(_CJK_RE.findall(raw_query))
     if not cjk_text:
         return None
-    return _parse_cjk_text(index, cjk_text, fields)
-
-
-def _build_ast_cjk_query(
-    index: tantivy.Index,
-    ast: wc.ast.Node,
-    registry: wc.FieldRegistry,
-) -> tantivy.Query | None:
-    """Build the bigram clause of a QUERY-mode search from the parsed AST.
-
-    Same discipline as the fuzzy clause (see _try_parse_fuzzy_query): the CJK
-    runs come from whoosh_compat's ``free_text_tokens`` over the parsed tree,
-    never from the raw query string, so a term the user negated or restricted
-    to a field outside the default search fields contributes nothing, instead
-    of resurfacing as a top-level clause matching every bigram field.
-
-    ``free_text_tokens`` reports no field of its own, so the tokens are
-    collected one default field at a time: a bare term, which the parser has
-    already copied onto every default field, is therefore searched across
-    every bigram field, while ``title:東京`` reaches ``bigram_title`` alone.
-    Fields whose CJK text is identical (the bare-term case) share a single
-    parse over all of their bigram fields at once.
-
-    Raw (``analyzed=False``) tokens are used because the bigram fields have
-    their own character-ngram analyzer: the default fields' word analyzers
-    have no useful say over a CJK run, and running them first would only
-    risk dropping it (remove_long) before the run is ever extracted.
-    Returns None when the query has no CJK free text.
-    """
-    fields_by_text: dict[str, list[str]] = {}
-    for field, bigram_field in _CJK_BIGRAM_FIELDS.items():
-        tokens = wc.free_text_tokens(
-            ast,
-            registry=registry,
-            fields=[field],
-            analyzed=False,
-        )
-        cjk_text = extract_cjk_text(" ".join(tokens))
-        if cjk_text:
-            fields_by_text.setdefault(cjk_text, []).append(bigram_field)
-
-    clauses: list[tuple[tantivy.Occur, tantivy.Query]] = [
-        (tantivy.Occur.Should, query)
-        for cjk_text, bigram_fields in fields_by_text.items()
-        if (query := _parse_cjk_text(index, cjk_text, bigram_fields)) is not None
-    ]
-    return _any_of(clauses) if clauses else None
-
-
-# A joined fuzzy word string must stay plain words: it goes back through
-# tantivy's own query parser, and the raw query text the clause collects
-# routinely carries characters that parser reads as grammar (a colon, a
-# bracket, a quote, a leading -). Each token is cut into its word runs and
-# only those are kept, so no field syntax, pattern, range or grouping can
-# reach the parser. Cutting rather than dropping the whole token is what
-# keeps ordinary hyphenated, dotted and quoted input ("COVID-19",
-# "hello@example.com", "tax reports") contributing to the clause at all.
-_WORD_RUN_RE = regex.compile(r"\w+")
-
-# The one piece of tantivy grammar that survives the cut: its boolean
-# keywords are themselves word runs. Only these exact spellings are
-# grammar there ("And"/"and" are ordinary terms), so lowercasing exactly
-# these turns them back into the ordinary terms the field analyzer used to
-# make of them, before the clause switched to raw text. Left alone, a
-# quoted phrase would silently restructure the clause ("tax AND reports"
-# becoming a conjunction) or fail to parse and drop it entirely
-# ("tax AND", or "IN" anywhere).
-#
-# Only these words are touched: tantivy lowercases query terms with the
-# field's own analyzer, and doing it ourselves first is not always the
-# same operation (Python folds a final sigma to a different letter than
-# tantivy does, and turns Turkish 'İ' into a sequence tantivy then splits
-# in two), which would search for terms the index does not contain.
-_TANTIVY_KEYWORDS: Final[frozenset[str]] = frozenset({"AND", "OR", "NOT", "IN"})
-
-
-def _try_parse_fuzzy_query(
-    index: tantivy.Index,
-    ast: wc.ast.Node,
-    registry: wc.FieldRegistry,
-) -> tantivy.Query | None:
-    """Build the fuzzy blend clause from the parsed query's free-text
-    words, or None if it has none.
-
-    The clause is built by handing tantivy's own query parser a plain
-    word string (there's no clean AST-level fuzzy equivalent to
-    whoosh-compat's parse tree, and fuzzy matching was always an
-    approximate, secondary, 0.1-boosted clause). The words come from
-    whoosh_compat's ``free_text_tokens`` over the already-parsed AST,
-    never from the raw query string: raw whoosh grammar (date keywords,
-    ``[2005 to 2009]`` ranges, bracket-class wildcards) is not tantivy
-    syntax, and feeding it here used to knock the fuzzy clause out for
-    the whole query the moment any such construct appeared alongside a
-    typo'd word. The helper also keeps excluded terms out: a ``NOT``'d
-    word must not resurface through the fuzzy clause.
-
-    Chosen trade-off: a term explicitly fielded on one of the default
-    search fields (``correspondent:acme``) contributes its text to the
-    word string UNFIELDED, so the fuzzy clause searches it across all
-    default fields rather than just the one the user named. That is
-    recall-only widening on a secondary 0.1-boosted clause the score
-    threshold already disciplines, accepted in exchange for never feeding
-    field syntax to tantivy's parser. What the word string guarantees is
-    exactly that: no field prefix, pattern, range, grouping or quoting
-    survives, and the boolean keywords that do survive (they are word
-    runs) are lowercased into ordinary terms; see _TANTIVY_KEYWORDS.
-
-    The words are the query's RAW text, not the analyzer's output
-    (``analyzed=False``), because ``index.parse_query`` analyzes whatever
-    it is given and analysis is not idempotent: ``universities`` stems to
-    ``univers``, and handing that back stems it again to ``univ``, a term
-    the index does not contain. ``prefix=True`` hid this as over-broad
-    matching (``univ`` also prefixes ``unicycle``) rather than as no
-    matches at all. Raw text is untokenized, which is why it is cut into
-    word runs above rather than taken whole.
-
-    The ValueError guard stays as insurance (the word string is plain
-    tokens, so tantivy accepting it is expected, not assumed): on a parse
-    failure the fuzzy clause is skipped and the exact/CJK clauses stand,
-    rather than the whole query failing.
-    """
-    tokens = wc.free_text_tokens(
-        ast,
-        registry=registry,
-        fields=_DEFAULT_SEARCH_FIELDS,
-        analyzed=False,
-    )
-    words = list(
-        dict.fromkeys(
-            word.lower() if word in _TANTIVY_KEYWORDS else word
-            for token in tokens
-            for word in _WORD_RUN_RE.findall(token)
-        ),
-    )
-    if not words:
-        return None
-    fuzzy_text = " ".join(words)
     try:
-        return index.parse_query(
-            fuzzy_text,
-            _DEFAULT_SEARCH_FIELDS,
-            field_boosts=_FIELD_BOOSTS,
-            fuzzy_fields={f: (True, 1, True) for f in _DEFAULT_SEARCH_FIELDS},
-        )
-    except ValueError:
-        logger.debug(
-            "Skipping fuzzy search clause: token string is not valid "
-            "tantivy query syntax: %r",
-            fuzzy_text,
-        )
+        return index.parse_query(cjk_text, fields)
+    except Exception:
         return None
 
 
-_DEFAULT_SEARCH_FIELDS: Final[list[str]] = [
+def build_permission_filter(
+    schema: tantivy.Schema,
+    user: AbstractBaseUser,
+    viewer_group_ids: Iterable[int] = (),
+) -> tantivy.Query:
+    """
+    Build a query filter for user document permissions.
+
+    Creates a query that matches only documents visible to the specified user
+    according to paperless-ngx permission rules:
+    - Public documents (no owner) are visible to all users
+    - Private documents are visible to their owner
+    - Documents explicitly shared with the user are visible
+    - Documents shared with one of the user's current groups are visible
+
+    Args:
+        schema: Tantivy schema for field validation
+        user: User to check permissions for
+        viewer_group_ids: Current group memberships for the user
+
+    Returns:
+        Tantivy query that filters results to visible documents
+    """
+    owner_any = tantivy.Query.exists_query("owner_id")
+    no_owner = tantivy.Query.boolean_query(
+        [
+            (tantivy.Occur.Must, tantivy.Query.all_query()),
+            (tantivy.Occur.MustNot, owner_any),
+        ],
+    )
+    owned = tantivy.Query.term_query(schema, "owner_id", user.pk)
+    shared = tantivy.Query.term_query(schema, "viewer_id", user.pk)
+    group_shared = [
+        tantivy.Query.term_query(schema, "viewer_group_id", group_id)
+        for group_id in viewer_group_ids
+    ]
+    return tantivy.Query.disjunction_max_query(
+        [no_owner, owned, shared, *group_shared],
+    )
+
+
+DEFAULT_SEARCH_FIELDS = [
     "title",
     "content",
     "correspondent",
     "document_type",
     "tag",
 ]
-_SIMPLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title", "simple_content"]
-_TITLE_SEARCH_FIELDS: Final[list[str]] = ["simple_title"]
-# The bigram (character-ngram) companion of each default search field.
-_CJK_BIGRAM_FIELDS: Final[dict[str, str]] = {
-    field: f"bigram_{field}" for field in _DEFAULT_SEARCH_FIELDS
-}
+SIMPLE_SEARCH_FIELDS = ["simple_title", "simple_content"]
+TITLE_SEARCH_FIELDS = ["simple_title"]
+_CJK_ALL_FIELDS: Final[list[str]] = [
+    "bigram_content",
+    "bigram_title",
+    "bigram_correspondent",
+    "bigram_document_type",
+    "bigram_tag",
+]
 _CJK_CONTENT_FIELDS: Final[list[str]] = ["bigram_content"]
 _CJK_TITLE_FIELDS: Final[list[str]] = ["bigram_title"]
 _FIELD_BOOSTS = {"title": 2.0}
 _SIMPLE_FIELD_BOOSTS = {"simple_title": 2.0}
 
 
-class _ConjunctiveNegations(wc.ast.Visitor[tuple["wc.ast.Node", ...]]):
-    """Collect the subtrees an AST excludes from every document it matches.
-
-    A negation reached through ``And``/``AndNot``/``Require`` (and through
-    the required half of an ``AndMaybe``) constrains the whole query, so it
-    can be re-stated above the blend. ``Or`` is deliberately not descended
-    into: in ``invoice OR NOT secret`` the negation is one branch's own
-    condition, and hoisting it would throw away documents the other branch
-    matches. Nor is a collected subtree descended into, since a negation
-    inside a negation is not an exclusion.
-
-    Node types with no negation to contribute (every leaf, ``Or``) fall
-    through to ``generic_visit``.
-    """
-
-    def generic_visit(self, node: wc.ast.Node) -> tuple[wc.ast.Node, ...]:
-        return ()
-
-    def visit_not(self, node: wc.ast.Not) -> tuple[wc.ast.Node, ...]:
-        return (node.child,)
-
-    def visit_andnot(self, node: wc.ast.AndNot) -> tuple[wc.ast.Node, ...]:
-        return (*self.visit(node.positive), node.negative)
-
-    def visit_and(self, node: wc.ast.And) -> tuple[wc.ast.Node, ...]:
-        return tuple(
-            negation for child in node.children for negation in self.visit(child)
-        )
-
-    def visit_boosted(self, node: wc.ast.Boosted) -> tuple[wc.ast.Node, ...]:
-        return self.visit(node.child)
-
-    def visit_andmaybe(self, node: wc.ast.AndMaybe) -> tuple[wc.ast.Node, ...]:
-        return self.visit(node.required)
-
-    def visit_require(self, node: wc.ast.Require) -> tuple[wc.ast.Node, ...]:
-        return (*self.visit(node.scored), *self.visit(node.filter_only))
-
-
-def _negation_clauses(
-    index: tantivy.Index,
-    ast: wc.ast.Node,
-    registry: wc.FieldRegistry,
-) -> list[tuple[tantivy.Occur, tantivy.Query]]:
-    """MustNot clauses for everything ``ast`` excludes conjunctively.
-
-    Each excluded subtree is emitted as its own positive query and attached
-    with ``MustNot``, rather than emitting a negative query and hoping
-    tantivy accepts a bare one.
-    """
-    try:
-        return [
-            (
-                tantivy.Occur.MustNot,
-                tantivy_emit(negation, index=index, registry=registry),
-            )
-            for negation in _ConjunctiveNegations().visit(ast)
-        ]
-    except QueryError as e:
-        raise _map_emit_error(e) from e
-
-
-def _any_of(clauses: list[tuple[tantivy.Occur, tantivy.Query]]) -> tantivy.Query:
-    """Collapse a clause list: none -> empty, one -> itself (no wasted
-    single-clause boolean_query wrapping), many -> boolean_query(clauses)."""
-    if not clauses:
-        return tantivy.Query.empty_query()
-    if len(clauses) == 1:
-        return clauses[0][1]
-    return tantivy.Query.boolean_query(clauses)
+def _simple_query_tokens(raw_query: str) -> list[str]:
+    # Tokenize and fold via the same analyzer used to index simple_title /
+    # simple_content, so query terms fold identically to the indexed terms
+    # (single source of truth for ASCII folding).
+    return simple_search_tokens(raw_query)
 
 
 def _build_simple_token_query(
@@ -424,7 +168,9 @@ def _build_simple_token_query(
             query = tantivy.Query.boost_query(query, boost)
         field_queries.append((tantivy.Occur.Should, query))
 
-    return _any_of(field_queries)
+    if len(field_queries) == 1:
+        return field_queries[0][1]
+    return tantivy.Query.boolean_query(field_queries)
 
 
 def parse_user_query(
@@ -433,53 +179,52 @@ def parse_user_query(
     tz: tzinfo,
 ) -> tantivy.Query:
     """
-    Parse user query through whoosh-compat, then blend in fuzzy/CJK clauses.
+    Parse user query through the complete preprocessing pipeline.
 
-    1. wc.parse() against the shared FieldRegistry (whoosh grammar -> AST).
-       Bare notes:/custom_fields: prefixes resolve to their default subpath
-       (notes.note:/custom_fields.value:) directly in the registry, via
-       each JSON field's SubpathSpec(default=True).
-    2. Any diagnostics (bad dates/numbers) map to SearchQueryError subclasses
-       and raise, the view returns HTTP 400 with every offending field
-       listed, not just the first.
-    3. emit() turns the AST into a tantivy.Query directly (no string
-       round-trip). A QueryError is routed by its Diagnostic's Cause
-       (_map_emit_error): a construct that parses but can't execute against
-       tantivy (e.g. a text-field range) is a 400, a registry/schema
-       mismatch is logged and a 400, and an INTERNAL defect is re-raised.
-    4. Optional fuzzy blend (ADVANCED_FUZZY_SEARCH_THRESHOLD) builds a
-       plain word string from the parsed AST's free-text tokens
-       (whoosh_compat.free_text_tokens) and feeds THAT to
-       index.parse_query, never raw_query, whose whoosh grammar (date
-       keywords, bracket-class wildcards, etc.) tantivy's parser rejects,
-       which used to silently knock the fuzzy clause out of any mixed
-       query (see _try_parse_fuzzy_query).
-    5. Optional CJK bigram clause, built from the same parsed AST for the
-       same reason (see _build_ast_cjk_query): a CJK term the query negated
-       or fielded must not resurface through it.
-    6. When any optional clause was added, the query's conjunctive
-       exclusions are restated as MustNot above the blend
-       (_negation_clauses): a clause built from positive terms cannot
-       express them, and as a bare Should it would undo them.
+    Transforms the raw user query through multiple stages:
+    1. Date keyword rewriting (today → ISO 8601 ranges)
+    2. Query normalization (comma expansion, whitespace cleanup)
+    3. Tantivy parsing with field boosts
+    4. Optional fuzzy query blending (if ADVANCED_FUZZY_SEARCH_THRESHOLD set)
+
+    Args:
+        index: Tantivy index with registered tokenizers
+        raw_query: Original user query string
+        tz: Timezone for date boundary calculations
+
+    Returns:
+        Parsed Tantivy query ready for execution
+
+    Note:
+        When ADVANCED_FUZZY_SEARCH_THRESHOLD is configured, adds a low-priority
+        fuzzy query as a Should clause (0.1 boost) to catch approximate matches
+        while keeping exact matches ranked higher. The threshold value is applied
+        as a post-search score filter, not during query construction.
     """
-    registry = get_field_registry(settings.SEARCH_LANGUAGE)
-    result = wc.parse(
-        raw_query,
-        registry=registry,
-        default_fields=_DEFAULT_SEARCH_FIELDS,
-        field_boosts=_FIELD_BOOSTS,
-        tz=tz,
-    )
-    if result.diagnostics:
-        raise _diagnostics_to_error(result.diagnostics)
 
     try:
-        exact = tantivy_emit(result.ast, index=index, registry=registry)
-    except QueryError as e:
-        raise _map_emit_error(e) from e
+        query_str = translate_query(raw_query, tz)
+    except SearchQueryError:
+        # Intentional, user-fixable error (e.g. an unparsable date). Propagate so
+        # the view can return a 400 with a helpful message rather than falling
+        # back to the raw (still-invalid) query.
+        raise
+    except Exception:  # pragma: no cover - defensive
+        logger.warning("Query translation failed; using raw query", exc_info=True)
+        query_str = raw_query
 
+    exact = index.parse_query(
+        query_str,
+        DEFAULT_SEARCH_FIELDS,
+        field_boosts=_FIELD_BOOSTS,
+    )
+
+    # The standard analyzer keeps a whitespace-free CJK run as a single token,
+    # so substring queries can't match content/title (and long runs are dropped
+    # by remove_long). Route CJK queries to the bigram fields, whose ngram
+    # tokenizer indexes overlapping 2-grams for substring matching.
     cjk_query = (
-        _build_ast_cjk_query(index, result.ast, registry)
+        _build_cjk_query(index, raw_query, _CJK_ALL_FIELDS)
         if _has_cjk(raw_query)
         else None
     )
@@ -490,73 +235,22 @@ def parse_user_query(
 
     threshold = settings.ADVANCED_FUZZY_SEARCH_THRESHOLD
     if threshold is not None:
-        fuzzy = _try_parse_fuzzy_query(index, result.ast, registry)
-        if fuzzy is not None:
-            clauses.append(
-                (tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)),
-            )
+        fuzzy = index.parse_query(
+            query_str,
+            DEFAULT_SEARCH_FIELDS,
+            field_boosts=_FIELD_BOOSTS,
+            # (prefix=True, distance=1, transposition_cost_one=True) — edit-distance fuzziness
+            fuzzy_fields={f: (True, 1, True) for f in DEFAULT_SEARCH_FIELDS},
+        )
+        # 0.1 boost keeps fuzzy hits ranked below exact matches (intentional)
+        clauses.append((tantivy.Occur.Should, tantivy.Query.boost_query(fuzzy, 0.1)))
 
     if cjk_query is not None:
         clauses.append((tantivy.Occur.Should, cjk_query))
 
     if len(clauses) == 1:
         return exact
-    # The fuzzy and CJK clauses are built from positive terms only, so as
-    # plain Shoulds beside the exact clause they re-admit exactly the
-    # documents the query excluded. Restate the exclusions once, above the
-    # whole blend. Redundant against the exact clause, which already
-    # carries them, but idempotently so.
-    negations = _negation_clauses(index, result.ast, registry)
-    if not negations:
-        return _any_of(clauses)
-    return tantivy.Query.boolean_query(
-        [(tantivy.Occur.Must, _any_of(clauses)), *negations],
-    )
-
-
-# The three whoosh-compat kinds for a wildcard on a field that cannot
-# carry one. d.field_kind supplies the discriminator, so naming the field's
-# type needs no second trip through the registry.
-_PATTERN_ON_KINDS: Final = frozenset(
-    {
-        DiagnosticKind.PATTERN_ON_NUMERIC,
-        DiagnosticKind.PATTERN_ON_BOOLEAN_EXISTS,
-        DiagnosticKind.PATTERN_ON_SUBPATH,
-    },
-)
-
-
-def _diagnostics_to_error(diagnostics: tuple[Diagnostic, ...]) -> SearchQueryError:
-    errors = [_single_diagnostic_to_error(d) for d in diagnostics]
-    return errors[0] if len(errors) == 1 else MultipleSearchQueryErrors(errors)
-
-
-def _single_diagnostic_to_error(d: Diagnostic) -> SearchQueryError:
-    # d.field is a FieldRef, not a str: str(d.field) gives the canonical
-    # dotted name (an aliased query, e.g. type:, reports document_type).
-    field_name = str(d.field) if d.field is not None else None
-    if d.kind is DiagnosticKind.BAD_DATE:
-        return InvalidDateQuery(field_name, d.raw_value)
-    if d.kind is DiagnosticKind.BAD_NUMBER:
-        return InvalidNumberQuery(field_name, d.raw_value)
-    if d.kind is DiagnosticKind.TOO_DEEP:
-        return SearchQueryError("The search query is nested too deeply.")
-    if d.kind in _PATTERN_ON_KINDS:
-        kind_label = f" ({d.field_kind.name.lower()})" if d.field_kind else ""
-        return SearchQueryError(
-            f"Wildcard patterns are not supported for field "
-            f"{field_name!r}{kind_label}.",
-        )
-    if d.kind is DiagnosticKind.SINGLE_CHAR_BRACKET_RANGE:
-        field_label = f" for field {field_name!r}" if field_name else ""
-        return SearchQueryError(
-            f"{d.raw_value!r} looks like a bracket range{field_label}, but "
-            "'[' is not a wildcard character on its own. Combine it with a "
-            "wildcard, e.g. a trailing '*', or double-quote the value to "
-            "search it as literal text.",
-        )
-    logger.warning("Unmapped parse diagnostic %s: %s", d.kind, d.message)
-    return SearchQueryError("The search query could not be executed.")
+    return tantivy.Query.boolean_query(clauses)
 
 
 def parse_simple_query(
@@ -574,7 +268,7 @@ def parse_simple_query(
     CJK substrings the simple analyzer can't (long whitespace-free runs are
     dropped by remove_long).
     """
-    tokens = simple_search_tokens(raw_query)
+    tokens = _simple_query_tokens(raw_query)
 
     clauses: list[tuple[tantivy.Occur, tantivy.Query]] = []
     if tokens:
@@ -597,14 +291,23 @@ def parse_simple_query(
             )
             for token in tokens
         ]
-        clauses.append((tantivy.Occur.Should, _any_of(token_queries)))
+        simple_query = (
+            token_queries[0][1]
+            if len(token_queries) == 1
+            else tantivy.Query.boolean_query(token_queries)
+        )
+        clauses.append((tantivy.Occur.Should, simple_query))
 
     if cjk_fields and _has_cjk(raw_query):
         cjk_q = _build_cjk_query(index, raw_query, cjk_fields)
         if cjk_q is not None:
             clauses.append((tantivy.Occur.Should, cjk_q))
 
-    return _any_of(clauses)
+    if not clauses:
+        return tantivy.Query.empty_query()
+    if len(clauses) == 1:
+        return clauses[0][1]
+    return tantivy.Query.boolean_query(clauses)
 
 
 def parse_simple_text_highlight_query(
@@ -619,21 +322,13 @@ def parse_simple_text_highlight_query(
 
     # Strip Tantivy operator chars before tokenizing: this is a plain-text
     # highlight query, not a structured boolean query, so +/- are separators.
-    tokens = simple_search_tokens(
+    tokens = _simple_query_tokens(
         regex.sub(r"[-+]", " ", raw_query, timeout=_REGEX_TIMEOUT),
     )
     if not tokens:
         return tantivy.Query.empty_query()
 
-    # Quote each token as its own phrase, escaping backslashes and embedded
-    # quotes. simple search tokens can carry arbitrary Tantivy syntax
-    # characters (`"`, `:`, `(`, `[`, `/`, ...) that the query-string parser
-    # would otherwise interpret as query grammar rather than literal text.
-    quoted_tokens = [
-        '"' + token.replace("\\", "\\\\").replace('"', '\\"') + '"' for token in tokens
-    ]
-
-    return index.parse_query(" ".join(quoted_tokens), ["content"])
+    return index.parse_query(" ".join(tokens), ["content"])
 
 
 def parse_simple_text_query(
@@ -647,7 +342,7 @@ def parse_simple_text_query(
     return parse_simple_query(
         index,
         raw_query,
-        _SIMPLE_SEARCH_FIELDS,
+        SIMPLE_SEARCH_FIELDS,
         cjk_fields=_CJK_CONTENT_FIELDS,
     )
 
@@ -663,6 +358,6 @@ def parse_simple_title_query(
     return parse_simple_query(
         index,
         raw_query,
-        _TITLE_SEARCH_FIELDS,
+        TITLE_SEARCH_FIELDS,
         cjk_fields=_CJK_TITLE_FIELDS,
     )

src/documents/search/_registry.py

@@ -1,91 +0,0 @@
-from __future__ import annotations
-
-import dataclasses
-from typing import TYPE_CHECKING
-
-from whoosh_compat import FieldKind
-from whoosh_compat import FieldRegistry
-
-from documents.search._fields import PUBLIC_FIELDS
-from documents.search._tokenizer import ascii_fold
-from documents.search._tokenizer import paperless_text_analyzer
-from documents.search._tokenizer import stem_pattern_text
-
-if TYPE_CHECKING:
-    from whoosh_compat import PatternNormalizer
-
-_registry_cache: dict[str | None, FieldRegistry] = {}
-
-
-def _identity_analyzer(text: str) -> list[str]:
-    """Analyzer for KEYWORD fields indexed with the raw tokenizer (no splitting)."""
-    return [text]
-
-
-def _fold_normalizer(text: str) -> str:
-    """Wildcard/regex literal-run normalizer for fields indexed without stemming."""
-    return ascii_fold(text.lower())
-
-
-def _make_pattern_normalizer(language: str | None) -> PatternNormalizer:
-    """Build the wildcard/regex literal-run normalizer for a search language."""
-
-    def _pattern_normalizer(text: str) -> tuple[str, ...]:
-        """Normalize a literal run into the forms a term may match.
-
-        TEXT index terms go through lowercase -> ascii_fold -> stem, so a
-        pattern that skips stemming can never match one: "invoice*" would look
-        for a term starting with "invoice" while the index holds "invoic". The
-        run is therefore offered stemmed as well. KEYWORD fields are indexed
-        raw and get _fold_normalizer instead, so their patterns stay literal.
-
-        Both forms are returned, as alternatives, because neither is a prefix
-        of the other in general: English stemming substitutes as well as
-        truncates ("copy" -> "copi"), so the stem alone loses the compounds
-        the typed run reaches ("copyright") while the typed run alone loses
-        the inflections the stem reaches ("copies"). whoosh-compat ORs the
-        alternatives per literal run and deduplicates them, so a run the
-        stemmer leaves alone costs exactly the one branch it did before.
-
-        Inside a bracket class the emitter calls this once per character and
-        uses the answer only if it is a single one-character form; two forms
-        there leave the character as typed. A stemmer does not change a lone
-        character, so the two forms deduplicate to one and the class body is
-        folded as before.
-        """
-        folded = ascii_fold(text.lower())
-        stemmed = stem_pattern_text(folded, language)
-        return (folded, stemmed)
-
-    return _pattern_normalizer
-
-
-def get_field_registry(language: str | None) -> FieldRegistry:
-    """Build (or return the cached) FieldRegistry for the given search language.
-
-    Cached keyed by language, rebuilt on the same trigger register_tokenizers()
-    uses (settings.SEARCH_LANGUAGE change). A fresh call with a new language
-    builds and caches a new registry rather than mutating the old one.
-    """
-    if language in _registry_cache:
-        return _registry_cache[language]
-
-    text_analyzer = paperless_text_analyzer(language).analyze
-    pattern_normalizer = _make_pattern_normalizer(language)
-
-    specs = [
-        dataclasses.replace(
-            field,
-            analyzer=_identity_analyzer
-            if field.kind is FieldKind.KEYWORD
-            else text_analyzer,
-            pattern_normalizer=_fold_normalizer
-            if field.kind is FieldKind.KEYWORD
-            else pattern_normalizer,
-        )
-        for field in PUBLIC_FIELDS
-    ]
-
-    registry = FieldRegistry(specs)
-    _registry_cache[language] = registry
-    return registry

src/documents/search/_schema.py

@@ -1,19 +1,14 @@
 from __future__ import annotations
 
-import hashlib
 import json
 import logging
 import shutil
 from typing import TYPE_CHECKING
 from typing import Final
-from typing import NamedTuple
 from typing import cast
 
 import tantivy
 from django.conf import settings
-from whoosh_compat import FieldKind
-
-from documents.search._fields import PUBLIC_FIELDS
 
 if TYPE_CHECKING:
     from pathlib import Path
@@ -21,185 +16,7 @@ if TYPE_CHECKING:
 logger = logging.getLogger("paperless.search")
 
 # v1 - Initial tantivy schema format
-# v2 - build_schema() derived from PUBLIC_FIELDS, changing the field declaration
-#      order, and the write-only correspondent/document_type/storage_path/tag id
-#      columns dropped. tantivy compares schemas by ordered field list, so an
-#      index built by v1 rejects every write against the v2 schema.
-SCHEMA_VERSION: Final[int] = 2
-
-
-class FieldDescriptor(NamedTuple):
-    """One tantivy field, in declaration order.
-
-    The descriptor vocabulary is paperless', not tantivy-py's: it is both the
-    input to the SchemaBuilder and the input to schema_fingerprint(), so the
-    persisted fingerprint cannot move under a tantivy-py upgrade.
-    """
-
-    name: str
-    kind: str
-    stored: bool
-    indexed: bool
-    fast: bool
-    tokenizer: str | None
-
-
-# (schema kind, tokenizer) for the FieldKind -> FieldDescriptor mapping that
-# doesn't need special-casing. JSON is handled separately below since it can
-# emit a second, synthetic descriptor.
-_KIND_TABLE: Final[dict[FieldKind, tuple[str, str | None]]] = {
-    FieldKind.TEXT: ("text", "paperless_text"),
-    FieldKind.KEYWORD: ("text", "raw"),
-    FieldKind.U64: ("u64", None),
-    FieldKind.DATE: ("date", None),
-    FieldKind.DATETIME: ("date", None),
-}
-# Kinds whose fast-field flag follows FieldSpec.fast rather than always False.
-_FAST_FROM_FIELD: Final[frozenset[FieldKind]] = frozenset(
-    {FieldKind.U64, FieldKind.DATE, FieldKind.DATETIME},
-)
-
-
-def _public_field_descriptors() -> list[FieldDescriptor]:
-    """Descriptors for the query-visible fields declared in PUBLIC_FIELDS."""
-    descriptors: list[FieldDescriptor] = []
-    for field in PUBLIC_FIELDS:
-        if field.kind is FieldKind.JSON:
-            descriptors.append(
-                FieldDescriptor(
-                    field.name,
-                    "json",
-                    stored=True,
-                    indexed=True,
-                    fast=False,
-                    tokenizer="paperless_text",
-                ),
-            )
-            if field.name == "notes":
-                # Plain-text companion for snippet generation: tantivy's
-                # SnippetGenerator does not support JSON fields. Schema-only,
-                # no query-syntax meaning, not in PUBLIC_FIELDS.
-                descriptors.append(
-                    FieldDescriptor(
-                        "notes_text",
-                        "text",
-                        stored=True,
-                        indexed=True,
-                        fast=False,
-                        tokenizer="paperless_text",
-                    ),
-                )
-            continue
-        schema_kind, tokenizer = _KIND_TABLE[field.kind]
-        descriptors.append(
-            FieldDescriptor(
-                field.name,
-                schema_kind,
-                stored=True,
-                indexed=True,
-                fast=field.fast if field.kind in _FAST_FROM_FIELD else False,
-                tokenizer=tokenizer,
-            ),
-        )
-    return descriptors
-
-
-def field_descriptors() -> list[FieldDescriptor]:
-    """Every field of the document index, in the order tantivy declares them.
-
-    tantivy compares schemas by *ordered* field list, so the order here is
-    part of the on-disk contract: schema_fingerprint() hashes it and
-    needs_rebuild() acts on the result.
-    """
-    return [
-        FieldDescriptor(
-            "id",
-            "u64",
-            stored=True,
-            indexed=True,
-            fast=True,
-            tokenizer=None,
-        ),
-        *_public_field_descriptors(),
-        # Shadow sort fields - fast, not stored
-        *(
-            FieldDescriptor(
-                name,
-                "text",
-                stored=False,
-                indexed=True,
-                fast=True,
-                tokenizer="simple_analyzer",
-            )
-            for name in ("title_sort", "correspondent_sort", "type_sort")
-        ),
-        # CJK support - not stored, indexed only
-        *(
-            FieldDescriptor(
-                name,
-                "text",
-                stored=False,
-                indexed=True,
-                fast=False,
-                tokenizer="bigram_analyzer",
-            )
-            for name in (
-                "bigram_content",
-                "bigram_title",
-                "bigram_correspondent",
-                "bigram_document_type",
-                "bigram_tag",
-            )
-        ),
-        # Simple substring search support for title/content - not stored,
-        # indexed only
-        *(
-            FieldDescriptor(
-                name,
-                "text",
-                stored=False,
-                indexed=True,
-                fast=False,
-                tokenizer="simple_search_analyzer",
-            )
-            for name in ("simple_title", "simple_content")
-        ),
-        # Autocomplete prefix scan via terms_with_prefix, which walks the
-        # field's term dictionary - so the field must be indexed (term dict),
-        # not stored. The stored value is never read back, so storing it only
-        # wastes space.
-        FieldDescriptor(
-            "autocomplete_word",
-            "text",
-            stored=False,
-            indexed=True,
-            fast=False,
-            tokenizer="raw",
-        ),
-        # Permission filter columns, read by build_permission_filter.
-        *(
-            FieldDescriptor(
-                name,
-                "u64",
-                stored=False,
-                indexed=True,
-                fast=True,
-                tokenizer=None,
-            )
-            for name in ("owner_id", "viewer_id", "viewer_group_id")
-        ),
-    ]
-
-
-def schema_fingerprint() -> str:
-    """Hash of the field descriptors, stamped into .index_settings.json.
-
-    Changes whenever a field is added, removed, retyped, re-optioned or
-    reordered, so an index built from a different schema shape is detected
-    even when SCHEMA_VERSION was not bumped.
-    """
-    payload = json.dumps([list(descriptor) for descriptor in field_descriptors()])
-    return hashlib.blake2b(payload.encode()).hexdigest()
+SCHEMA_VERSION: Final[int] = 1
 
 
 def build_schema() -> tantivy.Schema:
@@ -215,37 +32,85 @@ def build_schema() -> tantivy.Schema:
     """
     sb = tantivy.SchemaBuilder()
 
-    for descriptor in field_descriptors():
-        if descriptor.kind == "text":
-            sb.add_text_field(
-                descriptor.name,
-                stored=descriptor.stored,
-                fast=descriptor.fast,
-                tokenizer_name=cast("str", descriptor.tokenizer),
-            )
-        elif descriptor.kind == "json":
-            sb.add_json_field(
-                descriptor.name,
-                stored=descriptor.stored,
-                fast=descriptor.fast,
-                tokenizer_name=cast("str", descriptor.tokenizer),
-            )
-        elif descriptor.kind == "u64":
-            sb.add_unsigned_field(
-                descriptor.name,
-                stored=descriptor.stored,
-                indexed=descriptor.indexed,
-                fast=descriptor.fast,
-            )
-        elif descriptor.kind == "date":
-            sb.add_date_field(
-                descriptor.name,
-                stored=descriptor.stored,
-                indexed=descriptor.indexed,
-                fast=descriptor.fast,
-            )
-        else:
-            raise ValueError(f"Unknown schema field kind: {descriptor.kind}")
+    sb.add_unsigned_field("id", stored=True, indexed=True, fast=True)
+    sb.add_text_field("checksum", stored=True, tokenizer_name="raw")
+
+    for field in (
+        "title",
+        "correspondent",
+        "document_type",
+        "storage_path",
+        "original_filename",
+        "content",
+    ):
+        sb.add_text_field(field, stored=True, tokenizer_name="paperless_text")
+
+    # Shadow sort fields - fast, not stored/indexed
+    for field in ("title_sort", "correspondent_sort", "type_sort"):
+        sb.add_text_field(
+            field,
+            stored=False,
+            tokenizer_name="simple_analyzer",
+            fast=True,
+        )
+
+    # CJK support - not stored, indexed only
+    sb.add_text_field("bigram_content", stored=False, tokenizer_name="bigram_analyzer")
+    sb.add_text_field("bigram_title", stored=False, tokenizer_name="bigram_analyzer")
+    sb.add_text_field(
+        "bigram_correspondent",
+        stored=False,
+        tokenizer_name="bigram_analyzer",
+    )
+    sb.add_text_field(
+        "bigram_document_type",
+        stored=False,
+        tokenizer_name="bigram_analyzer",
+    )
+    sb.add_text_field("bigram_tag", stored=False, tokenizer_name="bigram_analyzer")
+
+    # Simple substring search support for title/content - not stored, indexed only
+    sb.add_text_field(
+        "simple_title",
+        stored=False,
+        tokenizer_name="simple_search_analyzer",
+    )
+    sb.add_text_field(
+        "simple_content",
+        stored=False,
+        tokenizer_name="simple_search_analyzer",
+    )
+
+    # Autocomplete prefix scan via terms_with_prefix, which walks the field's
+    # term dictionary - so the field must be indexed (term dict), not stored.
+    # The stored value is never read back, so storing it only wastes space.
+    sb.add_text_field("autocomplete_word", stored=False, tokenizer_name="raw")
+
+    sb.add_text_field("tag", stored=True, tokenizer_name="paperless_text")
+
+    # JSON fields — structured queries: notes.user:alice, custom_fields.name:invoice
+    sb.add_json_field("notes", stored=True, tokenizer_name="paperless_text")
+    # Plain-text companion for notes — tantivy's SnippetGenerator does not support
+    # JSON fields, so highlights require a text field with the same content.
+    sb.add_text_field("notes_text", stored=True, tokenizer_name="paperless_text")
+    sb.add_json_field("custom_fields", stored=True, tokenizer_name="paperless_text")
+
+    for field in (
+        "correspondent_id",
+        "document_type_id",
+        "storage_path_id",
+        "tag_id",
+        "owner_id",
+        "viewer_id",
+        "viewer_group_id",
+    ):
+        sb.add_unsigned_field(field, stored=False, indexed=True, fast=True)
+
+    for field in ("created", "modified", "added"):
+        sb.add_date_field(field, stored=True, indexed=True, fast=True)
+
+    for field in ("asn", "page_count", "num_notes"):
+        sb.add_unsigned_field(field, stored=True, indexed=True, fast=True)
 
     return sb.build()
 
@@ -254,9 +119,9 @@ def needs_rebuild(index_dir: Path) -> bool:
     """
     Check if the search index needs rebuilding.
 
-    Reads .index_settings.json to compare the stored schema version, search
-    language and schema fingerprint against the current configuration. Returns
-    True if the file is missing, unparsable, or any value mismatches.
+    Reads .index_settings.json to compare the stored schema version and
+    search language against the current configuration. Returns True if the
+    file is missing, unparsable, or either value mismatches.
 
     Args:
         index_dir: Path to the search index directory
@@ -275,9 +140,6 @@ def needs_rebuild(index_dir: Path) -> bool:
         if "language" not in data or data["language"] != settings.SEARCH_LANGUAGE:
             logger.info("Search index language changed - rebuilding.")
             return True
-        if data.get("schema_fingerprint") != schema_fingerprint():
-            logger.info("Search index schema fingerprint mismatch - rebuilding.")
-            return True
     except ValueError:
         return True
     return False
@@ -308,7 +170,6 @@ def _write_sentinels(index_dir: Path) -> None:
             {
                 "schema_version": SCHEMA_VERSION,
                 "language": settings.SEARCH_LANGUAGE,
-                "schema_fingerprint": schema_fingerprint(),
             },
         ),
     )

src/documents/search/_tokenizer.py

@@ -1,7 +1,6 @@
 from __future__ import annotations
 
 import logging
-from functools import cache
 from typing import Final
 
 import tantivy
@@ -72,7 +71,7 @@ def register_tokenizers(index: tantivy.Index, language: str | None) -> None:
         use fast=True and Tantivy requires fast-field tokenizers to exist
         even for documents that omit those fields.
     """
-    index.register_tokenizer("paperless_text", paperless_text_analyzer(language))
+    index.register_tokenizer("paperless_text", _paperless_text(language))
     index.register_tokenizer("simple_analyzer", _simple_analyzer())
     index.register_tokenizer("bigram_analyzer", _bigram_analyzer())
     index.register_tokenizer("simple_search_analyzer", _simple_search_analyzer())
@@ -80,7 +79,7 @@ def register_tokenizers(index: tantivy.Index, language: str | None) -> None:
     index.register_fast_field_tokenizer("simple_analyzer", _simple_analyzer())
 
 
-def paperless_text_analyzer(language: str | None) -> tantivy.TextAnalyzer:
+def _paperless_text(language: str | None) -> tantivy.TextAnalyzer:
     """Main full-text tokenizer for content, title, etc: simple -> remove_long(129) -> lowercase -> ascii_fold [-> stemmer]"""
     builder = (
         tantivy.TextAnalyzerBuilder(tantivy.Tokenizer.simple())
@@ -101,54 +100,6 @@ def paperless_text_analyzer(language: str | None) -> tantivy.TextAnalyzer:
     return builder.build()
 
 
-@cache
-def _pattern_stemmer(language: str | None) -> tantivy.TextAnalyzer | None:
-    """The stemming tail of paperless_text_analyzer, over a whole literal run.
-
-    Same language gate and same Snowball stemmer paperless_text_analyzer
-    applies at index time, so query patterns follow SEARCH_LANGUAGE. Returns
-    None when that gate disables stemming; paperless_text_analyzer already
-    warns about an unsupported language, so this stays quiet.
-
-    The raw tokenizer keeps the run whole (a wildcard literal is a fragment,
-    not necessarily a word), and remove_long is kept so an over-long run is
-    treated the same way the index treats it.
-    """
-    if not language:
-        return None
-    tantivy_lang = _LANGUAGE_MAP.get(language.lower())
-    if tantivy_lang is None:
-        return None
-    return (
-        tantivy.TextAnalyzerBuilder(tantivy.Tokenizer.raw())
-        .filter(tantivy.Filter.remove_long(_TOKEN_REMOVE_LONG_LIMIT))
-        .filter(tantivy.Filter.stemmer(tantivy_lang))
-        .build()
-    )
-
-
-def stem_pattern_text(text: str, language: str | None) -> str:
-    """Stem an already lowercased/ascii-folded run the way index terms are.
-
-    Returns text unchanged when stemming is disabled for language, and also
-    when the stem step does not yield exactly one token: remove_long drops a run
-    past the length limit, leaving no stem to substitute. Falling back to the
-    text as typed is the safe direction for a pattern prefix, since it can only
-    be as narrow as it was before stemming was considered.
-
-    The raw tokenizer emits one token whatever the input and the stemmer is
-    1-to-1, so only the zero-token case can fire today; the guard covers both
-    counts so a tokenizer change cannot turn this into an IndexError.
-    """
-    analyzer = _pattern_stemmer(language)
-    if analyzer is None:
-        return text
-    tokens = analyzer.analyze(text)
-    if len(tokens) != 1:
-        return text
-    return tokens[0]
-
-
 def _simple_analyzer() -> tantivy.TextAnalyzer:
     """Tokenizer for shadow sort fields (title_sort, correspondent_sort, type_sort): simple -> lowercase -> ascii_fold."""
     return (

src/documents/search/_translate.py

@@ -0,0 +1,610 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from datetime import UTC
+from datetime import datetime
+from datetime import timedelta
+from typing import TYPE_CHECKING
+from typing import TypeAlias
+
+import regex
+from dateutil.relativedelta import relativedelta
+
+from documents.search._dates import _DATE_KEYWORDS
+from documents.search._dates import _DATE_ONLY_FIELDS
+from documents.search._dates import _date_only_range
+from documents.search._dates import _datetime_range
+from documents.search._dates import _field_range_from_dates
+from documents.search._dates import _fmt
+from documents.search._dates import _precision_bounds
+from documents.search._dates import _utc_bounds_for_field
+
+# Compiled regex that matches any known multi-word (or single-word) date keyword
+# at the start of a match position, longest alternatives first so "previous week"
+# wins over a hypothetical shorter "previous".
+_KEYWORD_VALUE_RE = regex.compile(
+    "|".join(sorted((regex.escape(k) for k in _DATE_KEYWORDS), key=len, reverse=True)),
+    regex.IGNORECASE,
+)
+
+if TYPE_CHECKING:
+    from datetime import tzinfo
+
+# TODO: this module translates date queries into Tantivy *string* syntax, which
+# forces a workaround for something Tantivy's string parser cannot express on
+# date fields: open-ended ranges use far-past/far-future string sentinels
+# (OPEN_LO/OPEN_HI). These can be replaced with a real tantivy.Query object
+# (Query.range_query(..., None) for open bounds) once tantivy-py accepts Python
+# datetimes in range_query/term_query on Date fields. That support exists on
+# tantivy-py master (PRs #655 + #666) but postdates the pinned 0.26.0 wheel, so
+# it is blocked only on a published release > 0.26.0 and a dependency bump.
+# (Unparsable dates now raise InvalidDateQuery -> HTTP 400 rather than using a
+# no-match string sentinel.)
+
+# Fields that store exact, non-analyzed comma-joined tokens in the index and so
+# need explicit comma->AND expansion (Whoosh KEYWORD(commas=True) set).
+MULTI_VALUE_FIELDS = frozenset({"tag", "tag_id", "viewer_id"})
+
+# Date fields whose values/ranges get rewritten to RFC3339 Tantivy ranges.
+DATE_FIELDS = frozenset({"created", "modified", "added"})
+
+# Field aliases: Whoosh (v2) field names that were renamed in the Tantivy schema.
+# Preserved here so v2 queries using the old names continue to work without 400
+# errors instead of silently failing. Applied by _render to non-date field tokens.
+FIELD_ALIASES: dict[str, str] = {
+    "type": "document_type",
+    "type_id": "document_type_id",
+    "path": "storage_path",
+    "path_id": "storage_path_id",
+}
+
+# Known schema fields: a comma immediately followed by ``<known>:`` is a clause
+# separator. Restricting to known fields prevents URL-like ``http:`` misfires.
+KNOWN_FIELDS = frozenset(
+    {
+        "title",
+        "content",
+        "correspondent",
+        "document_type",
+        "type",  # v2 alias -> document_type
+        "storage_path",
+        "path",  # v2 alias -> storage_path
+        "tag",
+        "tag_id",
+        "correspondent_id",
+        "document_type_id",
+        "type_id",  # v2 alias -> document_type_id
+        "storage_path_id",
+        "path_id",  # v2 alias -> storage_path_id
+        "owner_id",
+        "viewer_id",
+        "asn",
+        "page_count",
+        "num_notes",
+        "created",
+        "modified",
+        "added",
+        "original_filename",
+        "checksum",
+        "notes",
+        "custom_fields",
+    },
+)
+
+_FIELD_RE = regex.compile(r"(?P<field>\w+):")
+
+# Matches the TO separator inside a range bracket. Handles three forms:
+#   middle:   "lo TO hi"   (either lo or hi may be empty)
+#   trailing: "lo TO"      (open upper bound)
+#   leading:  "TO hi"      (open lower bound)
+# Bounds MAY contain internal spaces (e.g. "-7 days"), so we use .*? / .+?
+# and split on the whitespace-delimited " TO " / " to " separator.
+_RANGE_RE = regex.compile(
+    r"^\s*(?P<lo>.*?)\s+[Tt][Oo]\s+(?P<hi>.+?)\s*$"
+    r"|"
+    r"^\s*(?P<lo2>.+?)\s+[Tt][Oo]\s*$"
+    r"|"
+    r"^\s*[Tt][Oo]\s+(?P<hi2>.+?)\s*$",
+)
+
+
+@dataclass(frozen=True, slots=True)
+class FieldValue:
+    field: str
+    value: str
+
+
+# Produced by the comma-resolution pass (not by scan()).
+@dataclass(frozen=True, slots=True)
+class FieldValueList:
+    field: str
+    values: tuple[str, ...]
+
+
+@dataclass(frozen=True, slots=True)
+class FieldRange:
+    field: str
+    open: str
+    lo: str
+    hi: str
+    close: str
+
+
+# Produced by the comma-resolution pass (not by scan()).
+@dataclass(frozen=True, slots=True)
+class Comma:
+    pass
+
+
+@dataclass(frozen=True, slots=True)
+class Passthrough:
+    raw: str
+
+
+Token: TypeAlias = FieldValue | FieldValueList | FieldRange | Comma | Passthrough
+
+_CLOSE: dict[str, str] = {"[": "]", "{": "}"}
+
+
+def scan(query: str) -> list[Token]:
+    """
+    Tokenize a raw query into date/comma-aware tokens, leaving everything else
+    as verbatim ``Passthrough`` runs. Non-recursive: finds the first matching
+    close bracket/quote. Nested brackets are not valid Tantivy range syntax and
+    pass through verbatim on mismatch.
+    """
+    tokens: list[Token] = []
+    buf: list[str] = []  # accumulates passthrough chars
+    i, n = 0, len(query)
+    while i < n:
+        matched = _match_field_token(query, i)
+        if matched is None:
+            buf.append(query[i])
+            i += 1
+            continue
+        token, i = matched
+        if buf and buf[-1] == ",":
+            buf.pop()
+            _flush(buf, tokens)
+            tokens.append(Comma())
+        else:
+            _flush(buf, tokens)
+        tokens.append(token)
+        i = _maybe_comma(query, i, tokens)
+    _flush(buf, tokens)
+    return tokens
+
+
+def _flush(buf: list[str], tokens: list[Token]) -> None:
+    """Emit any accumulated passthrough characters as a single token."""
+    if buf:
+        tokens.append(Passthrough("".join(buf)))
+        buf.clear()
+
+
+def _at_word_boundary(query: str, i: int) -> bool:
+    """A field token may begin only at the start or after a non-word character."""
+    return i == 0 or not (query[i - 1].isalnum() or query[i - 1] == "_")
+
+
+def _match_field_token(query: str, i: int) -> tuple[Token, int] | None:
+    """
+    If a known ``field:`` token starts at ``i``, consume it and return
+    ``(token, end_index)``; otherwise return None so the caller treats the
+    character as passthrough. Handles both ``field:[range]`` and ``field:value``,
+    and returns None when the range/value cannot be consumed.
+    """
+    m = _FIELD_RE.match(query, i)
+    if m is None or m.group("field") not in KNOWN_FIELDS:
+        return None
+    if not _at_word_boundary(query, i):
+        return None
+    field = m.group("field")
+    j = m.end()
+    if j < len(query) and query[j] in "[{":
+        return _consume_range(query, j, field)
+    consumed = _consume_field_value(query, field, j)
+    if consumed is None:
+        return None
+    value, end = consumed
+    return FieldValue(field, value), end
+
+
+def _consume_field_value(query: str, field: str, start: int) -> tuple[str, int] | None:
+    """
+    Consume a field value starting at ``start``: a multi-word date keyword phrase
+    (date fields only), or a bare/quoted value, then absorb any comma-joined
+    continuation that is not a clause separator. ``resolve_commas`` later splits a
+    multi-value field's joined value into a ``FieldValueList``; for other fields
+    the comma stays literal.
+    """
+    n = len(query)
+    consumed = None
+    if field in DATE_FIELDS:
+        km = _KEYWORD_VALUE_RE.match(query, start)
+        if km is not None and (km.end() >= n or query[km.end()] in " \t),"):
+            consumed = (km.group(0), km.end())
+    if consumed is None:
+        consumed = _consume_value(query, start)
+    if consumed is None:
+        return None
+    value, k = consumed
+    while k < n and query[k] == ",":
+        if _looks_like_known_field(query, k + 1):
+            break  # clause separator: left for _maybe_comma to emit a Comma()
+        more = _consume_value(query, k + 1)
+        if more is None:
+            break
+        value = f"{value},{more[0]}"
+        k = more[1]
+    return value, k
+
+
+def _consume_range(
+    query: str,
+    start: int,
+    field: str,
+) -> tuple[FieldRange, int] | None:
+    """Consume ``[lo TO hi]`` / ``{lo TO hi}`` from ``start`` (the bracket)."""
+    open_br = query[start]
+    close_br = _CLOSE[open_br]
+    end = query.find(close_br, start + 1)
+    if end == -1:
+        return None
+    inner = query[start + 1 : end]
+    m = _RANGE_RE.match(inner)
+    if m is not None:
+        if m.group("lo") is not None or m.group("hi") is not None:
+            # Middle form: "lo TO hi" (either may be empty string)
+            lo = (m.group("lo") or "").strip()
+            hi = (m.group("hi") or "").strip()
+        elif m.group("lo2") is not None:
+            # Trailing form: "lo TO"
+            lo = m.group("lo2").strip()
+            hi = ""
+        else:
+            # Leading form: "TO hi"
+            lo = ""
+            hi = (m.group("hi2") or "").strip()
+    else:
+        lo, hi = inner.strip(), ""
+    return FieldRange(field, open_br, lo, hi, close_br), end + 1
+
+
+def _consume_value(query: str, start: int) -> tuple[str, int] | None:
+    """Consume a bare or quoted field value from ``start``, stopping at comma."""
+    n = len(query)
+    if start >= n or query[start] in " \t":
+        return None
+    if query[start] in "\"'":
+        quote = query[start]
+        end = query.find(quote, start + 1)
+        if end == -1:
+            return None
+        return query[start : end + 1], end + 1
+    j = start
+    while j < n and query[j] not in " \t),":
+        j += 1
+    return query[start:j], j
+
+
+def _looks_like_known_field(query: str, pos: int) -> bool:
+    """True if a known ``field:`` token starts at ``pos``."""
+    m = _FIELD_RE.match(query, pos)
+    return bool(m and m.group("field") in KNOWN_FIELDS)
+
+
+def _maybe_comma(query: str, i: int, tokens: list) -> int:
+    """If a clause-separator comma follows at ``i``, emit ``Comma()`` and advance."""
+    if i < len(query) and query[i] == "," and _looks_like_known_field(query, i + 1):
+        tokens.append(Comma())
+        return i + 1
+    return i
+
+
+def resolve_commas(tokens: list) -> list:
+    """
+    Collapse value-list commas into ``FieldValueList`` and keep clause-separator
+    commas as ``Comma``. (Clause-sep commas are already emitted by ``scan`` via
+    the value-stop logic; this pass folds value-lists.)
+    """
+    out: list = []
+    for tok in tokens:
+        if (
+            isinstance(tok, FieldValue)
+            and tok.field in MULTI_VALUE_FIELDS
+            and "," in tok.value
+        ):
+            values = tuple(v for v in tok.value.split(",") if v)
+            out.append(FieldValueList(tok.field, values))
+        else:
+            out.append(tok)
+    return out
+
+
+class SearchQueryError(ValueError):
+    """
+    Base for user-fixable search query errors.
+
+    Carries a message safe to surface to the user (no internal details). The view
+    layer catches this and returns an HTTP 400, so any future subclass (unknown
+    field, malformed range, wrapped parser errors) gets the same treatment.
+    """
+
+
+class InvalidDateQuery(SearchQueryError):
+    """Raised when a date field value or range bound cannot be parsed."""
+
+    def __init__(self, field: str, value: str) -> None:
+        self.field = field
+        self.value = value
+        super().__init__(f"Invalid date value {value!r} for field {field!r}.")
+
+
+_DIGITS_RE = regex.compile(r"^\d{4}(?:\d{2}){0,2}$")
+_ISO_RE = regex.compile(r"^\d{4}(?:-\d{2}(?:-\d{2})?)?$")
+
+
+def translate_scalar(field: str, value: str, tz: tzinfo) -> str:
+    """Translate a bare date-field value to a Tantivy range string."""
+    bare = value.strip("\"'").lower()
+    if bare in _DATE_KEYWORDS:
+        if field in _DATE_ONLY_FIELDS:
+            return f"{field}:{_date_only_range(bare, tz)}"
+        return f"{field}:{_datetime_range(bare, tz)}"
+    digits = value.replace("-", "")
+    if _DIGITS_RE.match(value) or _ISO_RE.match(value):
+        bounds = _precision_bounds(digits)
+        if bounds is None:
+            raise InvalidDateQuery(field, value)
+        return _field_range_from_dates(field, bounds[0], bounds[1], tz)
+    if regex.fullmatch(r"\d{14}", value):
+        try:
+            dt = datetime(
+                int(value[0:4]),
+                int(value[4:6]),
+                int(value[6:8]),
+                int(value[8:10]),
+                int(value[10:12]),
+                int(value[12:14]),
+                tzinfo=UTC,
+            )
+        except ValueError:
+            raise InvalidDateQuery(field, value) from None
+        iso = _fmt(dt)
+        return f"{field}:[{iso} TO {iso}]"
+    # Unrecognized shape -> tell the user their date is malformed rather than
+    # silently matching nothing or emitting invalid Tantivy syntax.
+    raise InvalidDateQuery(field, value)
+
+
+# Open-bound sentinels for date ranges. These far-past/far-future strings allow
+# open-ended ranges to be expressed as Tantivy string queries until tantivy-py
+# exposes Query.range_query(..., None) on Date fields (see module TODO).
+OPEN_LO = "0001-01-01T00:00:00Z"
+OPEN_HI = "9999-12-31T23:59:59Z"
+
+
+# Matches compact now-offset tokens like now-7d, now+1h, now-30m.
+_NOW_COMPACT_RE = regex.compile(
+    r"^now(?P<sign>[+-])(?P<n>\d+)(?P<unit>[dhm])$",
+    regex.IGNORECASE,
+)
+
+# Matches "±N <unit>" Whoosh-style offsets (e.g. -7 days, -1 week, +3 hours).
+# Whoosh's own date parser (qparser.dateparse.PlusMinus) additionally accepted
+# abbreviated unit spellings (e.g. "yrs", "yr", "y", "mos", "wks", "hrs", "mins",
+# "secs"); saved views/searches created under the old Whoosh backend can still
+# contain those tokens (e.g. "-999yrs"), so they are accepted here too and
+# normalized to a canonical unit via _UNIT_ALIASES below.
+_NOW_SPACED_RE = regex.compile(
+    r"^(?P<sign>[+-])(?P<n>\d+)\s*"
+    r"(?P<unit>years|year|yrs|yr|ys|y"
+    r"|months|month|mons|mon|mos|mo"
+    r"|weeks|week|wks|wk|ws|w"
+    r"|days|day|dys|dy|ds|d"
+    r"|hours|hour|hrs|hr|hs|h"
+    r"|minutes|minute|mins|min|ms|m"
+    r"|seconds|second|secs|sec|s)$",
+    regex.IGNORECASE,
+)
+
+# Maps every accepted unit spelling (including Whoosh-era abbreviations) to the
+# canonical unit name used as a key into the delta map in _resolve_relative_bound.
+_UNIT_ALIASES: dict[str, str] = {
+    alias: canonical
+    for canonical, aliases in {
+        "year": ("years", "year", "yrs", "yr", "ys", "y"),
+        "month": ("months", "month", "mons", "mon", "mos", "mo"),
+        "week": ("weeks", "week", "wks", "wk", "ws", "w"),
+        "day": ("days", "day", "dys", "dy", "ds", "d"),
+        "hour": ("hours", "hour", "hrs", "hr", "hs", "h"),
+        "minute": ("minutes", "minute", "mins", "min", "ms", "m"),
+        "second": ("seconds", "second", "secs", "sec", "s"),
+    }.items()
+    for alias in aliases
+}
+
+
+def _resolve_relative_bound(token: str) -> datetime | None:
+    """
+    Resolve a relative bound token to an exact UTC instant, or return None.
+
+    Supported forms:
+      - ``now``            -> current UTC instant
+      - ``now+/-<n>d/h/m`` -> now +/- timedelta (d=days, h=hours, m=minutes)
+      - ``±N <unit>``     -> now +/- delta; month/year use relativedelta;
+                              unit also accepts Whoosh-era abbreviations
+                              (e.g. "yrs", "mos", "wks", "hrs", "mins", "secs")
+    """
+    stripped = token.strip()
+    low = stripped.lower()
+    now = datetime.now(UTC)
+
+    if low == "now":
+        return now
+
+    m = _NOW_COMPACT_RE.match(stripped)
+    if m:
+        sign = 1 if m.group("sign") == "+" else -1
+        n = int(m.group("n"))
+        unit = m.group("unit").lower()
+        delta = (
+            sign
+            * {
+                "d": timedelta(days=n),
+                "h": timedelta(hours=n),
+                "m": timedelta(minutes=n),
+            }[unit]
+        )
+        return now + delta
+
+    m = _NOW_SPACED_RE.match(stripped)
+    if m:
+        sign = 1 if m.group("sign") == "+" else -1
+        n = int(m.group("n"))
+        unit = _UNIT_ALIASES[m.group("unit").lower()]
+        delta_map: dict[str, timedelta | relativedelta] = {
+            "second": timedelta(seconds=n),
+            "minute": timedelta(minutes=n),
+            "hour": timedelta(hours=n),
+            "day": timedelta(days=n),
+            "week": timedelta(weeks=n),
+            "month": relativedelta(months=n),
+            "year": relativedelta(years=n),
+        }
+        return now - delta_map[unit] if sign == -1 else now + delta_map[unit]
+
+    return None
+
+
+def _bound_datetimes(
+    field: str,
+    token: str,
+    tz: tzinfo,
+) -> tuple[datetime, datetime] | None:
+    """
+    Return (floor_dt, ceil_dt) UTC datetimes for a single range bound token, or
+    None if the token is unparsable. ``now`` and relative offsets resolve to the
+    current instant (floor == ceil == that instant; no day-flooring).
+    """
+    token = token.strip()
+
+    # Try relative/now forms first (before stripping hyphens which would mangle them).
+    rel = _resolve_relative_bound(token)
+    if rel is not None:
+        return rel, rel
+
+    # Full ISO datetime token (contains "T"): parse directly and return an exact
+    # instant (floor == ceil). Python 3.11+ datetime.fromisoformat accepts trailing Z.
+    if "T" in token:
+        try:
+            dt = datetime.fromisoformat(token)
+            # Ensure timezone-aware UTC result.
+            dt = dt.replace(tzinfo=UTC) if dt.tzinfo is None else dt.astimezone(UTC)
+            return dt, dt
+        except ValueError:
+            return None
+
+    digits = token.replace("-", "")
+    bounds = _precision_bounds(digits)
+    if bounds is None:
+        return None
+    start, end = bounds
+    return _utc_bounds_for_field(field, start, end, tz)
+
+
+def _render(tok: Token, tz: tzinfo) -> str:
+    """Render a single token back to a Tantivy query string fragment."""
+    if isinstance(tok, Passthrough):
+        return tok.raw
+    if isinstance(tok, Comma):
+        return " AND "
+    if isinstance(tok, FieldValueList):
+        field = FIELD_ALIASES.get(tok.field, tok.field)
+        return " AND ".join(f"{field}:{v}" for v in tok.values)
+    if isinstance(tok, FieldValue):
+        field = FIELD_ALIASES.get(tok.field, tok.field)
+        if field in DATE_FIELDS:
+            return translate_scalar(field, tok.value, tz)
+        return f"{field}:{tok.value}"
+    if isinstance(tok, FieldRange):
+        field = FIELD_ALIASES.get(tok.field, tok.field)
+        if field in DATE_FIELDS:
+            return translate_range(field, tok.lo, tok.hi, tz)
+        return f"{field}:{tok.open}{tok.lo} TO {tok.hi}{tok.close}"
+    return ""  # pragma: no cover
+
+
+# Post-render operator normalization patterns: collapse repeated whitespace and
+# strip spaced/trailing Tantivy boolean operators that would otherwise be invalid.
+_MULTI_SPACE_RE = regex.compile(r" {2,}")
+_TRAILING_OP_RE = regex.compile(r"\s+[-+]+\s*$")
+_SPACED_OP_RE = regex.compile(r"\s+[-+]\s+")
+
+
+def _normalize_operators(text: str) -> str:
+    """
+    Collapse multiple spaces, strip trailing dangling operators, and replace
+    spaced operators (`` - `` / `` + ``) with a single space.
+
+    Applied only to Passthrough fragments (the rendered output is scanned for
+    operator artifacts outside bracketed ranges) via a post-render pass on the
+    full rendered string. This preserves date ranges (``[... TO ...]``) verbatim
+    while cleaning natural-language separators in the surrounding text.
+    """
+    text = _MULTI_SPACE_RE.sub(" ", text)
+    text = _TRAILING_OP_RE.sub("", text).strip()
+    text = _SPACED_OP_RE.sub(" ", text).strip()
+    return text
+
+
+def translate_query(raw: str, tz: tzinfo) -> str:
+    """Translate a raw Whoosh-style query into Tantivy-compatible syntax."""
+    tokens = resolve_commas(scan(raw))
+    rendered = "".join(_render(t, tz) for t in tokens)
+    return _normalize_operators(rendered)
+
+
+def translate_range(field: str, lo: str, hi: str, tz: tzinfo) -> str:
+    """Translate a date-field ``[lo TO hi]`` range to a Tantivy ISO range string.
+
+    Handles partial-date bounds (YYYY, YYYYMM, YYYYMMDD, ISO dash variants),
+    open bounds (empty string -> OPEN_LO/OPEN_HI), ``now``, and reversed ranges
+    (swaps tokens before computing floor/ceil so the span is always correct).
+    """
+    lo_s = lo.strip()
+    hi_s = hi.strip()
+
+    # Parse both bounds to (floor, ceil) pairs when present.
+    lo_pair: tuple[datetime, datetime] | None = None
+    hi_pair: tuple[datetime, datetime] | None = None
+
+    if lo_s:
+        lo_pair = _bound_datetimes(field, lo_s, tz)
+        if lo_pair is None:
+            raise InvalidDateQuery(field, lo_s)
+    if hi_s:
+        hi_pair = _bound_datetimes(field, hi_s, tz)
+        if hi_pair is None:
+            raise InvalidDateQuery(field, hi_s)
+
+    # Detect a reversed range: only swap when BOTH bounds are present.
+    if lo_pair is not None and hi_pair is not None and lo_pair[0] > hi_pair[0]:
+        lo_pair, hi_pair = hi_pair, lo_pair
+
+    lo_iso = _fmt(lo_pair[0]) if lo_pair is not None else OPEN_LO
+
+    # A bound resolves to (floor, ceil) where floor == ceil for an exact instant
+    # (a full ISO datetime, "now", or a "+/-N unit" offset) and floor != ceil for
+    # a coarser period token (year/month/day precision). Only the latter needs a
+    # half-open close: its ceil is the start of the *next* period and must be
+    # excluded, or that instant (e.g. the 1st of next month) wrongly matches.
+    if hi_pair is not None:
+        hi_iso = _fmt(hi_pair[1])
+        hi_close = "]" if hi_pair[0] == hi_pair[1] else "}"
+    else:
+        hi_iso = OPEN_HI
+        hi_close = "]"
+
+    return f"{field}:[{lo_iso} TO {hi_iso}{hi_close}"

src/documents/serialisers.py

@@ -1795,7 +1795,7 @@ class DeleteDocumentsSerializer(DocumentSelectionSerializer):
 
 
 class ReprocessDocumentsSerializer(DocumentSelectionSerializer):
-    pass
+    remote_ocr = serializers.BooleanField(required=False, default=False)
 
 
 class BulkEditSerializer(
@@ -2137,6 +2137,13 @@ class BulkEditSerializer(
                         f"Page {op['page']} is out of bounds for document with {doc.page_count} pages.",
                     )
 
+    def _validate_parameters_reprocess(self, parameters) -> None:
+        if "remote_ocr" in parameters:
+            if not isinstance(parameters["remote_ocr"], bool):
+                raise serializers.ValidationError("remote_ocr must be a boolean")
+        else:
+            parameters["remote_ocr"] = False
+
     def validate_parameters_remove_password(self, parameters):
         if "password" not in parameters:
             raise serializers.ValidationError("password not specified")
@@ -2201,6 +2208,8 @@ class BulkEditSerializer(
             self._validate_parameters_edit_pdf(parameters, attrs["documents"][0])
         elif method == bulk_edit.remove_password:
             self.validate_parameters_remove_password(parameters)
+        elif method == bulk_edit.reprocess:
+            self._validate_parameters_reprocess(parameters)
 
         return attrs
 
@@ -3226,6 +3235,9 @@ class WorkflowActionSerializer(serializers.ModelSerializer[WorkflowAction]):
             "email",
             "webhook",
             "passwords",
+            "ai_suggestion_fields",
+            "ai_create_missing",
+            "ai_overwrite_existing",
         ]
 
     def validate(self, attrs):
@@ -3283,6 +3295,23 @@ class WorkflowActionSerializer(serializers.ModelSerializer[WorkflowAction]):
                     "Passwords are required for password removal actions",
                 )
 
+        if (
+            "type" in attrs
+            and attrs["type"] == WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
+        ):
+            fields = attrs.get("ai_suggestion_fields")
+            valid_fields = set(WorkflowAction.AISuggestionField.values)
+            if (
+                fields is None
+                or not isinstance(fields, list)
+                or len(fields) == 0
+                or any(field not in valid_fields for field in fields)
+            ):
+                raise serializers.ValidationError(
+                    "At least one valid field is required for apply AI "
+                    f"suggestions actions, options are: {sorted(valid_fields)}",
+                )
+
         return attrs
 
 
@@ -3303,6 +3332,68 @@ class WorkflowSerializer(serializers.ModelSerializer[Workflow]):
             "actions",
         ]
 
+    def validate(self, attrs):
+        attrs = super().validate(attrs)
+
+        if "actions" in attrs:
+            has_remote_ocr_action = any(
+                action.get("type") == WorkflowAction.WorkflowActionType.REMOTE_OCR
+                for action in attrs["actions"]
+            )
+            has_ai_suggestions_action = any(
+                action.get("type")
+                == WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
+                for action in attrs["actions"]
+            )
+        else:
+            has_remote_ocr_action = self.instance is not None and (
+                self.instance.actions.filter(
+                    type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
+                ).exists()
+            )
+            has_ai_suggestions_action = self.instance is not None and (
+                self.instance.actions.filter(
+                    type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
+                ).exists()
+            )
+
+        if "triggers" in attrs:
+            has_consumption_trigger = any(
+                trigger.get("type") == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
+                for trigger in attrs["triggers"]
+            )
+            has_non_consumption_trigger = any(
+                trigger.get("type") != WorkflowTrigger.WorkflowTriggerType.CONSUMPTION
+                for trigger in attrs["triggers"]
+            )
+        else:
+            has_consumption_trigger = self.instance is not None and (
+                self.instance.triggers.filter(
+                    type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+                ).exists()
+            )
+            has_non_consumption_trigger = self.instance is not None and (
+                self.instance.triggers.exclude(
+                    type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+                ).exists()
+            )
+
+        # Remote OCR can only work with consumption triggers
+        if has_remote_ocr_action and not has_consumption_trigger:
+            raise serializers.ValidationError(
+                "Remote OCR actions require a consumption started trigger",
+            )
+
+        # Suggestions are made from the document content, which does not exist
+        # until after consumption has finished
+        if has_ai_suggestions_action and not has_non_consumption_trigger:
+            raise serializers.ValidationError(
+                "Apply AI suggestions actions require a trigger other than "
+                "consumption started",
+            )
+
+        return attrs
+
     def update_triggers_and_actions(
         self,
         instance: Workflow,

src/documents/signals/handlers.py

@@ -973,6 +973,39 @@ def run_workflows(
                     )
                 elif action.type == WorkflowAction.WorkflowActionType.MOVE_TO_TRASH:
                     has_move_to_trash_action = True
+                elif action.type == WorkflowAction.WorkflowActionType.REMOTE_OCR:
+                    if use_overrides and overrides:
+                        overrides.remote_ocr = True
+                    else:
+                        # If a workflow has a consumption trigger *and* another type,
+                        # the document has already been parsed by the time the other one fires
+                        logger.debug(
+                            "Remote OCR action only applies to consumption "
+                            "triggers, ignoring",
+                            extra={"group": logging_group},
+                        )
+                elif (
+                    action.type
+                    == WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS
+                ):
+                    if use_overrides:
+                        # The document has not been parsed yet, so there is no
+                        # content for the LLM to make suggestions from
+                        logger.debug(
+                            "Apply AI suggestions action does not apply to "
+                            "consumption triggers, ignoring",
+                            extra={"group": logging_group},
+                        )
+                    else:
+                        # Queued rather than run sync
+                        from documents.tasks import apply_ai_suggestions
+
+                        # kwargs so the PaperlessTask record can note the
+                        # document, see _extract_input_data
+                        apply_ai_suggestions.delay(
+                            action_id=action.pk,
+                            document_id=document.pk,
+                        )
 
             if not use_overrides:
                 # limit title to 128 characters
@@ -1028,6 +1061,7 @@ TRACKED_TASKS: dict[str, PaperlessTask.TaskType] = {
     "documents.tasks.update_document_content_maybe_archive_file": PaperlessTask.TaskType.REPROCESS_DOCUMENT,
     "documents.tasks.build_share_link_bundle": PaperlessTask.TaskType.BUILD_SHARE_LINK,
     "documents.bulk_edit.delete": PaperlessTask.TaskType.BULK_DELETE,
+    "documents.tasks.apply_ai_suggestions": PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS,
 }
 
 _CELERY_STATE_TO_STATUS: dict[str, PaperlessTask.Status] = {
@@ -1081,6 +1115,12 @@ def _extract_input_data(
             return {"account_ids": account_ids}
         return {}
 
+    if task_type == PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS:
+        document_id = task_kwargs.get("document_id")
+        if document_id is not None:
+            return {"document_id": document_id}
+        return {}
+
     return {}
 
 

src/documents/tasks.py

@@ -67,9 +67,11 @@ from documents.utils import identity
 from documents.versioning import annotate_effective_content
 from documents.workflows.utils import get_workflows_for_trigger
 from paperless.config import AIConfig
+from paperless.config import RemoteOCRConfig
 from paperless.logging import consume_task_id
 from paperless.parsers import ParserContext
 from paperless.parsers.registry import get_parser_registry
+from paperless_ai.exceptions import LLMTimeoutError
 from paperless_ai.indexing import llm_index_add_or_update_document
 from paperless_ai.indexing import llm_index_remove_document
 from paperless_ai.indexing import update_llm_index
@@ -338,10 +340,17 @@ def bulk_update_documents(document_ids) -> None:
 
 
 @shared_task
-def update_document_content_maybe_archive_file(document_id) -> None:
+def update_document_content_maybe_archive_file(
+    document_id,
+    *,
+    remote_ocr: bool = False,
+) -> None:
     """
     Re-creates OCR content and thumbnail for a document, and archive file if
     it exists.
+
+    Remote OCR is used only when the engine is configured to handle everything
+    or if explicitly asked for via ``remote_ocr``.
     """
     document = Document.objects.get(id=document_id)
 
@@ -351,6 +360,7 @@ def update_document_content_maybe_archive_file(document_id) -> None:
         mime_type,
         document.original_filename or "",
         document.source_path,
+        allow_remote=remote_ocr or RemoteOCRConfig().remote_ocr_by_default,
     )
 
     if not parser_class:
@@ -705,6 +715,45 @@ def llmindex_index(
     )
 
 
+@shared_task(
+    bind=True,
+    autoretry_for=(LLMTimeoutError,),
+    max_retries=3,
+    retry_backoff=60,
+    retry_backoff_max=600,
+    retry_jitter=True,
+)
+def apply_ai_suggestions(self, action_id: int, document_id: int) -> None:
+    """
+    Deferred "apply AI suggestions" workflow action.
+    """
+    from documents.models import WorkflowAction
+    from documents.workflows.ai import apply_ai_suggestions_to_document
+
+    try:
+        action = WorkflowAction.objects.get(pk=action_id)
+        document = Document.objects.select_related("owner").get(pk=document_id)
+    except (WorkflowAction.DoesNotExist, Document.DoesNotExist):
+        logger.warning(
+            "Workflow action %s or document %s no longer exists, "
+            "not applying AI suggestions",
+            action_id,
+            document_id,
+        )
+        return
+
+    if not apply_ai_suggestions_to_document(action, document):
+        return
+
+    # No document_updated signal to avoid loop
+    clear_document_caches(document.pk)
+    index_document.delay(document.pk)
+
+    ai_config = AIConfig()
+    if ai_config.llm_index_enabled:
+        update_document_in_llm_index.apply_async(kwargs={"document": document})
+
+
 @shared_task
 def update_document_in_llm_index(document) -> None:
     llm_index_add_or_update_document(document)

src/documents/tests/search/conftest.py

@@ -1,11 +1,15 @@
 from __future__ import annotations
 
+import tempfile
 from typing import TYPE_CHECKING
 
 import pytest
+import tantivy
 
 from documents.search._backend import TantivyBackend
 from documents.search._backend import reset_backend
+from documents.search._schema import build_schema
+from documents.search._tokenizer import register_tokenizers
 
 if TYPE_CHECKING:
     from collections.abc import Generator
@@ -31,3 +35,11 @@ def backend() -> Generator[TantivyBackend, None, None]:
     finally:
         b.close()
         reset_backend()
+
+
+@pytest.fixture(scope="module")
+def index() -> tantivy.Index:
+    """A real Tantivy index for parse-acceptance tests (module scope for speed)."""
+    idx = tantivy.Index(build_schema(), path=tempfile.mkdtemp())
+    register_tokenizers(idx, "english")
+    return idx

src/documents/tests/search/test_acceptance.py

@@ -1,411 +0,0 @@
-"""Result-level acceptance corpus: real documents indexed via build_schema(),
-real queries run through parse_user_query(), matched-document-ID sets
-asserted, not intermediate ASTs or query strings. This is paperless-ngx's
-analogue of whoosh-compat's own tests/emitter/test_acceptance_e2e.py.
-
-Supersedes test_query.py's TestParseUserQuery result-level cases.
-"""
-
-from __future__ import annotations
-
-from datetime import UTC
-from datetime import datetime
-from typing import TYPE_CHECKING
-
-import pytest
-import time_machine
-from django.contrib.auth.models import User
-
-from documents.models import CustomField
-from documents.models import CustomFieldInstance
-from documents.models import Document
-from documents.models import DocumentType
-from documents.models import Note
-from documents.models import StoragePath
-from documents.search._query import parse_user_query
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-FROZEN_NOW = datetime(2026, 6, 15, 12, 0, tzinfo=UTC)
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    """Create a Document and index it in one step, for the common case
-    where nothing needs to happen between the two (no related Note/
-    CustomFieldInstance to attach first)."""
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-@pytest.fixture
-def indexed_documents(backend: TantivyBackend) -> dict[str, int]:
-    """Index a small fixture set, return {label: doc_id} for corpus queries."""
-    docs = {
-        "invoice_2020": _index(
-            backend,
-            title="Invoice 2020",
-            content="invoice total due",
-            checksum="acc-invoice-2020",
-            archive_serial_number=100,
-        ),
-        "invoice_2021": _index(
-            backend,
-            title="Invoice 2021",
-            content="invoice total due",
-            checksum="acc-invoice-2021",
-            archive_serial_number=101,
-        ),
-        "invoice_2023": _index(
-            backend,
-            title="Invoice 2023",
-            content="invoice total due",
-            checksum="acc-invoice-2023",
-            archive_serial_number=102,
-        ),
-        "receipt_2022": _index(
-            backend,
-            title="Receipt 2022",
-            content="receipt total due",
-            checksum="acc-receipt-2022",
-            archive_serial_number=103,
-        ),
-    }
-    return {label: doc.pk for label, doc in docs.items()}
-
-
-class TestIssue13568BracketWildcard:
-    """paperless-ngx#13568: title:202[0-3]* must keep its character class,
-    not fold to a prefix query that silently drops it."""
-
-    def test_bracket_class_wildcard_matches_only_in_range_years(
-        self,
-        backend: TantivyBackend,
-        indexed_documents: dict[str, int],
-    ) -> None:
-        # [0-1] (not [0-3]) is deliberate: the fixture's four years are
-        # 2020/2021/2022/2023, i.e. their trailing digit is 0/1/2/3
-        # respectively - a [0-3] class would match all four and the test
-        # would pass even if the character class were silently dropped and
-        # folded to an unconstrained "202*" prefix. [0-1] partitions the
-        # fixture into a genuine in-range/out-of-range split.
-        matched = _matched_ids(backend, "title:202[0-1]*")
-        expected = {
-            indexed_documents["invoice_2020"],
-            indexed_documents["invoice_2021"],
-        }
-        assert matched == expected, (
-            "title:202[0-1]* must match 2020/2021 titles and exclude 2022/2023 "
-            "- if this matches everything, the wildcard's character class was "
-            "silently dropped (issue #13568's original bug)"
-        )
-
-
-class TestFieldBoosts:
-    def test_title_boost_ranks_title_match_above_content_only_match(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        title_match = _index(
-            backend,
-            title="urgent",
-            content="nothing else relevant",
-            checksum="acc-boost-title",
-        )
-        _index(
-            backend,
-            title="nothing",
-            content="urgent matter here",
-            checksum="acc-boost-content",
-        )
-        query = parse_user_query(backend._index, "urgent", UTC)
-        searcher = backend._index.searcher()
-        results = searcher.search(query, limit=10)
-        ranked_ids = [
-            searcher.doc(addr).to_dict()["id"][0] for _score, addr in results.hits
-        ]
-        assert ranked_ids[0] == title_match.pk
-
-
-class TestJsonSubpaths:
-    def test_notes_user_matches_document_with_that_note_author(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        alice = User.objects.create_user(username="alice")
-        doc_with_note = Document.objects.create(
-            title="Has note",
-            content="x",
-            checksum="acc-note-with",
-        )
-        Note.objects.create(document=doc_with_note, user=alice, note="reminder")
-        backend.add_or_update(doc_with_note)
-        _index(backend, title="No note", content="x", checksum="acc-note-without")
-        matched = _matched_ids(backend, "notes.user:alice")
-        assert matched == {doc_with_note.pk}
-
-    def test_custom_fields_name_and_value_combine(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        field = CustomField.objects.create(
-            name="Contract Number",
-            data_type=CustomField.FieldDataType.STRING,
-        )
-        other_field = CustomField.objects.create(
-            name="Other Field",
-            data_type=CustomField.FieldDataType.STRING,
-        )
-        matching = Document.objects.create(
-            title="Matching",
-            content="x",
-            checksum="acc-cf-matching",
-        )
-        CustomFieldInstance.objects.create(
-            document=matching,
-            field=field,
-            value_text="policy",
-        )
-        backend.add_or_update(matching)
-        non_matching = Document.objects.create(
-            title="Non-matching",
-            content="x",
-            checksum="acc-cf-nonmatching",
-        )
-        CustomFieldInstance.objects.create(
-            document=non_matching,
-            field=other_field,
-            value_text="policy",
-        )
-        backend.add_or_update(non_matching)
-        matched = _matched_ids(
-            backend,
-            'custom_fields.name:"Contract Number" custom_fields.value:policy',
-        )
-        assert matched == {matching.pk}
-
-
-class TestUnregisteredIdFieldFoldsToLiteralText:
-    """tag_id, owner_id, etc. are intentionally excluded from the
-    FieldRegistry - always internal index columns, never meant to be
-    query-addressable. Prove an unregistered field folds to a literal
-    text search that matches nothing, rather than erroring."""
-
-    def test_tag_id_query_matches_nothing(
-        self,
-        backend: TantivyBackend,
-        indexed_documents: dict[str, int],
-    ) -> None:
-        matched = _matched_ids(backend, "tag_id:5")
-        assert matched == set()
-
-
-class TestFuzzyBlendSurvivesWhooshGrammar:
-    """A query mixing whoosh-only grammar (a date keyword) with a typo'd
-    free-text word must still fuzzy-match the intended document when
-    ADVANCED_FUZZY_SEARCH_THRESHOLD is enabled. The fuzzy clause is built
-    from the parsed query's free-text tokens (whoosh_compat's
-    free_text_tokens), never from the raw query string, so whoosh grammar
-    that tantivy's own parser rejects cannot knock the fuzzy clause out."""
-
-    def test_typo_fuzzy_matches_alongside_date_keyword(
-        self,
-        backend: TantivyBackend,
-        settings,
-    ) -> None:
-        settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.5
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            doc = _index(
-                backend,
-                title="Receipt March",
-                content="receipt total due",
-                checksum="fuzzy-blend-1",
-                archive_serial_number=900,
-            )
-            # Sanity: the exact spelling matches through the exact clause.
-            assert doc.pk in _matched_ids(backend, "added:today receipt")
-            # The regression: the misspelling (one transposition) only
-            # matches via the fuzzy clause, and "added:today" is
-            # whoosh-only grammar tantivy's parser rejects, so raw-string
-            # fuzzy parsing skips the clause entirely and this returns
-            # nothing. The typo is deliberate; keep codespell away from it.
-            typo_query = "added:today reciept"  # codespell:ignore reciept
-            assert doc.pk in _matched_ids(backend, typo_query)
-
-    def test_negated_words_do_not_fuzzy_match(
-        self,
-        backend: TantivyBackend,
-        settings,
-    ) -> None:
-        # A term the user excluded must not resurface through the fuzzy
-        # clause. The shape is chosen so this genuinely discriminates: the
-        # indexed document contains the NOT'd word but NOT the positive
-        # word, so nothing matches the exact clause, and a fuzzy string
-        # naively built from ALL words (including the NOT'd one) would
-        # make this document the sole hit, normalize its score to 1.0,
-        # and survive any threshold. (A shape with an exact-matching
-        # sibling document does NOT discriminate: normalization ranks the
-        # resurfaced doc far below the exact match and the threshold cuts
-        # it even for a naive implementation.)
-        settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.5
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            _index(
-                backend,
-                title="Receipt Archive",
-                content="receipt archived stack",
-                checksum="fuzzy-blend-2",
-                archive_serial_number=901,
-            )
-            assert _matched_ids(backend, "added:today total NOT receipt") == set()
-
-
-class TestUnquotedDateKeywordPhrases:
-    """The unquoted spelling (added:previous month) is honored natively by
-    whoosh-compat's own grammar for this closed phrase vocabulary, no
-    app-level rewrite is involved. Pins that the historically supported
-    spelling keeps working now that paperless no longer pre-quotes it."""
-
-    @pytest.fixture
-    def period_documents(self, backend: TantivyBackend) -> dict[str, int]:
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            in_may = _index(
-                backend,
-                title="May Doc",
-                content="statement",
-                checksum="kw-may",
-                archive_serial_number=910,
-                added=datetime(2026, 5, 20, 12, 0, tzinfo=UTC),
-            )
-            in_june = _index(
-                backend,
-                title="June Doc",
-                content="statement",
-                checksum="kw-june",
-                archive_serial_number=911,
-                added=datetime(2026, 6, 10, 12, 0, tzinfo=UTC),
-            )
-        return {"in_may": in_may.pk, "in_june": in_june.pk}
-
-    @pytest.mark.parametrize(
-        "query",
-        [
-            pytest.param("added:previous month", id="unquoted"),
-            pytest.param('added:"previous month"', id="quoted"),
-            pytest.param("added:Previous Month", id="unquoted-mixed-case"),
-        ],
-    )
-    def test_unquoted_matches_the_same_documents_as_quoted(
-        self,
-        backend: TantivyBackend,
-        period_documents: dict[str, int],
-        query: str,
-    ) -> None:
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            assert _matched_ids(backend, query) == {period_documents["in_may"]}
-
-    @pytest.mark.parametrize(
-        "query",
-        [
-            pytest.param("added:this month", id="this-month"),
-            pytest.param("added:this year", id="this-year"),
-            pytest.param("added:previous week", id="previous-week"),
-            pytest.param("added:previous quarter", id="previous-quarter"),
-            pytest.param("added:previous year", id="previous-year"),
-            pytest.param("created:previous month", id="created-field"),
-            pytest.param("modified:previous month", id="modified-field"),
-        ],
-    )
-    def test_every_phrase_and_date_field_parses_without_error(
-        self,
-        backend: TantivyBackend,
-        period_documents: dict[str, int],
-        query: str,
-    ) -> None:
-        # The whole vocabulary times every date field must at least parse
-        # and search cleanly (no SearchQueryError -> no HTTP 400); exact
-        # window semantics are whoosh-compat's, pinned in its own suite.
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            _matched_ids(backend, query)
-
-    def test_text_field_keyword_words_are_ordinary_text(
-        self,
-        backend: TantivyBackend,
-        period_documents: dict[str, int],
-    ) -> None:
-        # "previous month" after a TEXT field (or unfielded) is ordinary
-        # text, not a date phrase: a title actually containing the words
-        # matches, and the date-window documents do not.
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            wordy = _index(
-                backend,
-                title="Notes from the previous month",
-                content="meeting notes",
-                checksum="kw-text",
-                archive_serial_number=912,
-            )
-            assert _matched_ids(backend, "title:previous month") == {wordy.pk}
-
-
-class TestFieldAliases:
-    """type:/path: are registry aliases for document_type:/storage_path:.
-    The only other alias coverage is parse-shape; these prove resolution
-    end-to-end against a real index."""
-
-    def test_type_alias_and_canonical_name_match_the_same_document(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        invoice_type = DocumentType.objects.create(name="invoice")
-        # Discriminating shape: document_type is itself a default search
-        # field, so if alias resolution ever broke and "type:invoice"
-        # demoted to unfielded text, the token would STILL match the typed
-        # document through the field value. The decoy carries the query
-        # word in content, so a demoted search matches BOTH documents and
-        # the exact-set assertions fail. (The title avoids stemming to
-        # "type": english stems Typed -> type.)
-        typed = _index(
-            backend,
-            title="First",
-            content="quarterly statement",
-            checksum="alias-type-1",
-            document_type=invoice_type,
-        )
-        _index(
-            backend,
-            title="Second",
-            content="invoice mentioned in body",
-            checksum="alias-type-2",
-        )
-        assert _matched_ids(backend, "type:invoice") == {typed.pk}
-        assert _matched_ids(backend, "document_type:invoice") == {typed.pk}
-
-    def test_path_alias_and_canonical_name_match_the_same_document(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        archive = StoragePath.objects.create(name="archive", path="archive/{title}")
-        stored = _index(
-            backend,
-            title="Stored",
-            content="quarterly statement",
-            checksum="alias-path-1",
-            storage_path=archive,
-        )
-        # storage_path is NOT a default search field today, so a demoted
-        # "path:archive" already matches nothing; the content decoy keeps
-        # this test discriminating even if it ever joins the defaults.
-        _index(
-            backend,
-            title="Loose",
-            content="archive mentioned in body",
-            checksum="alias-path-2",
-        )
-        assert _matched_ids(backend, "path:archive") == {stored.pk}
-        assert _matched_ids(backend, "storage_path:archive") == {stored.pk}

src/documents/tests/search/test_checksum_prefix_queries.py

@@ -1,57 +0,0 @@
-"""``checksum`` wildcard patterns stay literal end to end, once user queries
-route through whoosh-compat.
-
-The registry-level fact (the pattern normalizer folds a KEYWORD pattern
-rather than stemming it) is pinned on its own in
-``test_keyword_pattern_literal.py``. This proves it actually reaches a real
-query: ``checksum:ceded*`` must match only the document whose checksum
-starts with "ceded", not the one whose checksum stems to the same run.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-CEDEF00D = "cedef00ddeadbeef0123456789abcdef01234567"
-CEDEDEAD = "cededeadbeef567801234567" + "89abcdef01234567"
-
-
-class TestChecksumPrefixQueries:
-    @pytest.fixture
-    def indexed(self, backend: TantivyBackend) -> None:
-        for i, checksum in enumerate((CEDEF00D, CEDEDEAD)):
-            doc = Document.objects.create(
-                title=f"Checksum doc {i}",
-                content="invoices for the quarter",
-                checksum=checksum,
-                archive_serial_number=940 + i,
-            )
-            backend.add_or_update(doc)
-
-    def _ids(self, backend: TantivyBackend, query: str) -> set[int]:
-        return set(backend.search_ids(query, user=None))
-
-    def test_prefix_matches_only_the_document_that_starts_with_it(
-        self,
-        backend: TantivyBackend,
-        indexed: None,
-    ) -> None:
-        matched = self._ids(backend, "checksum:ceded*")
-        expected = Document.objects.get(checksum=CEDEDEAD).pk
-        assert matched == {expected}
-
-    def test_text_prefix_still_reaches_the_stemmed_index(
-        self,
-        backend: TantivyBackend,
-        indexed: None,
-    ) -> None:
-        assert len(self._ids(backend, "invoice*")) == 2

src/documents/tests/search/test_cjk_clause.py

@@ -1,133 +0,0 @@
-"""The CJK bigram clause blended into QUERY-mode searches.
-
-The clause exists so CJK runs are matchable at all (the default analyzers
-keep a whitespace-free CJK run as one indivisible token), but it must not
-widen the query beyond what the user asked for: a CJK term the query
-excludes, or restricts to one field, must not come back through it.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from pytest_django.fixtures import SettingsWrapper
-
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-class TestCjkClauseFollowsTheParsedQuery:
-    def test_negated_cjk_term_is_excluded(self, backend: TantivyBackend) -> None:
-        """'invoice NOT 漢字' must not return the document containing 漢字."""
-        with_cjk = _index(
-            backend,
-            title="Invoice A",
-            content="invoice total 漢字",
-            checksum="cjk-neg-1",
-        )
-        without_cjk = _index(
-            backend,
-            title="Invoice B",
-            content="invoice total only",
-            checksum="cjk-neg-2",
-        )
-
-        assert _matched_ids(backend, "invoice") == {with_cjk.pk, without_cjk.pk}
-        assert _matched_ids(backend, "invoice NOT 漢字") == {without_cjk.pk}
-
-    @pytest.mark.parametrize(
-        ("threshold", "expected"),
-        [
-            pytest.param(None, {"titled"}, id="fuzzy_off"),
-            pytest.param(0.0, {"titled", "content_only"}, id="fuzzy_on"),
-        ],
-    )
-    def test_fielded_cjk_term_searches_only_that_field(
-        self,
-        backend: TantivyBackend,
-        settings: SettingsWrapper,
-        threshold: float | None,
-        expected: set[str],
-    ) -> None:
-        """'title:東京' must not match a document whose 東京 is in the content.
-
-        The CJK clause honours the field. The fuzzy clause, when enabled,
-        does not: it contributes every free-text term UNFIELDED by design
-        (see _try_parse_fuzzy_query), so it brings the content-only
-        document back on its own 0.1-boosted terms. That is the documented
-        trade-off, pinned here so it stays deliberate.
-        """
-        settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = threshold
-        content_only = _index(
-            backend,
-            title="Tokyo report",
-            content="東京都の人口は約1400万人です",
-            checksum="cjk-field-1",
-        )
-        titled = _index(
-            backend,
-            title="東京都の報告書",
-            content="an english summary",
-            checksum="cjk-field-2",
-        )
-        pks = {"titled": titled.pk, "content_only": content_only.pk}
-
-        assert _matched_ids(backend, "東京") == set(pks.values())
-        assert _matched_ids(backend, "title:東京") == {pks[label] for label in expected}
-
-    def test_cjk_on_a_non_default_field_builds_no_clause(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """A CJK term restricted to a field outside the default search fields
-        has nothing to contribute to the bigram clause: 'notes:東京' must not
-        fall back to matching 東京 in the content."""
-        _index(
-            backend,
-            title="Tokyo report",
-            content="東京都の人口は約1400万人です",
-            checksum="cjk-notes-1",
-        )
-
-        assert _matched_ids(backend, "notes:東京") == set()
-
-    def test_bare_cjk_term_still_matches_every_default_field(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """The clause's reason for existing: an unfielded CJK run matches
-        wherever it is indexed, and does so alongside a latin term."""
-        in_content = _index(
-            backend,
-            title="report",
-            content="本文に重要な情報",
-            checksum="cjk-bare-1",
-        )
-        in_title = _index(
-            backend,
-            title="重要な報告書",
-            content="english only",
-            checksum="cjk-bare-2",
-        )
-
-        assert _matched_ids(backend, "重要") == {in_content.pk, in_title.pk}
-        assert _matched_ids(backend, "重要 OR report") == {
-            in_content.pk,
-            in_title.pk,
-        }

src/documents/tests/search/test_date_keyword_phrase_removal.py

@@ -1,72 +0,0 @@
-"""Pins the correctness gained by deleting the pre-parse
-_quote_date_keyword_phrases rewrite.
-
-That rewrite matched date-keyword phrases (e.g. "previous month" after a
-date field) anywhere in the raw query string, including inside an
-unrelated quoted string, and inserted quotes mid-phrase there too. Its
-own docstring gave ``title:"see added:previous month notes"`` as the
-example of what it corrupted. whoosh-compat's grammar accepts the same
-phrase vocabulary unquoted natively (see TestUnquotedDateKeywordPhrases
-in test_acceptance.py), so the rewrite was redundant everywhere it was
-safe and actively wrong everywhere it was not. This is the one case that
-tells the two apart: a literal title phrase that happens to contain
-"added:previous month" as running text.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-class TestQuotedStringContainingDateKeywordText:
-    """A quoted title phrase containing the literal text
-    "added:previous month" as running words must match on that literal
-    text alone, never spill into an unfielded search for "previous" and
-    "month" across the default search fields the way the deleted rewrite
-    would have decomposed it into."""
-
-    def test_matches_only_the_literal_phrase(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        literal = _index(
-            backend,
-            title="see added:previous month notes",
-            content="quarterly filing",
-            checksum="dkp-literal",
-            archive_serial_number=920,
-        )
-        # Under the deleted rewrite, this decoy would incorrectly match:
-        # its title contains the "see added:" and " notes" fragments the
-        # corrupted parse required as title phrases, and its content
-        # supplies "previous" and "month" as the decomposed word-match
-        # clauses the rewrite turned the middle of the phrase into.
-        decoy = _index(
-            backend,
-            title="see added: quarterly report notes",
-            content="we reviewed the previous statement about month end",
-            checksum="dkp-decoy",
-            archive_serial_number=921,
-        )
-        query = 'title:"see added:previous month notes"'
-        assert _matched_ids(backend, query) == {literal.pk}
-        assert decoy.pk not in _matched_ids(backend, query)

src/documents/tests/search/test_date_keyword_timezone.py

@@ -1,83 +0,0 @@
-"""Date keyword phrases (``today``, etc.) resolved in a non-UTC timezone,
-end to end.
-
-paperless's own ``tz=get_current_timezone()`` plumbing
-(``TantivyBackend._parse_query``) is exercised elsewhere only for
-relative *ranges* (``added:[-1 week to now]``, in
-documents/tests/test_api_search.py). This covers a date *keyword*
-(``today``), whose day boundary depends on the active timezone the same
-way but goes through whoosh-compat's DateParserPlugin resolution instead
-of an explicit range.
-
-Discriminating shape: frozen at 2026-06-15T02:00 UTC, which is
-2026-06-14T22:00 in America/New_York -- still "today" (06-14) there, but
-already "today" (06-15) in UTC. Two documents pin both directions of the
-mistake a hardcoded-UTC bug would make:
-
-- ``in_ny_today`` (added 2026-06-14T20:00 UTC = 2026-06-14T16:00 NY) is
-  inside New York's "today" window and outside a naive UTC-calendar-day
-  window. A ``tz``-ignoring bug would miss it.
-- ``in_utc_calendar_day_only`` (added 2026-06-15T10:00 UTC =
-  2026-06-15T06:00 NY) is inside a naive UTC-calendar-day window but
-  outside New York's actual "today" window. A ``tz``-ignoring bug would
-  wrongly match it.
-"""
-
-from __future__ import annotations
-
-from datetime import UTC
-from datetime import datetime
-from typing import TYPE_CHECKING
-
-import pytest
-import time_machine
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from pytest_django.fixtures import SettingsWrapper
-
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-FROZEN_NOW = datetime(2026, 6, 15, 2, 0, tzinfo=UTC)
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-class TestDateKeywordUsesTheActiveTimezone:
-    def test_today_matches_the_new_york_calendar_day_not_the_utc_one(
-        self,
-        backend: TantivyBackend,
-        settings: SettingsWrapper,
-    ) -> None:
-        settings.TIME_ZONE = "America/New_York"
-        with time_machine.travel(FROZEN_NOW, tick=False):
-            in_ny_today = _index(
-                backend,
-                title="NY today",
-                content="x",
-                checksum="tz-keyword-ny-today",
-                added=datetime(2026, 6, 14, 20, 0, tzinfo=UTC),
-            )
-            # Not captured: the exact-set assertion below already proves
-            # this document (inside a naive UTC-calendar-day window, but
-            # outside New York's actual "today") does not match.
-            _index(
-                backend,
-                title="UTC calendar day only",
-                content="x",
-                checksum="tz-keyword-utc-calendar-day-only",
-                added=datetime(2026, 6, 15, 10, 0, tzinfo=UTC),
-            )
-
-            assert _matched_ids(backend, "added:today") == {in_ny_today.pk}

src/documents/tests/search/test_default_search_fields_guard.py

@@ -1,20 +0,0 @@
-"""``_DEFAULT_SEARCH_FIELDS`` must stay a subset of the registered public
-field names.
-
-Nothing enforced this before: a rename in PUBLIC_FIELDS not mirrored in
-``_DEFAULT_SEARCH_FIELDS`` (documents/search/_query.py) would 400 every
-unfielded search at request time, since ``index.parse_query`` and the
-fuzzy/CJK clause builders are handed a field name the schema no longer
-has.
-"""
-
-from __future__ import annotations
-
-from documents.search._fields import PUBLIC_FIELDS
-from documents.search._query import _DEFAULT_SEARCH_FIELDS
-
-
-class TestDefaultSearchFieldsAreRegistered:
-    def test_every_default_search_field_is_a_public_field(self) -> None:
-        public_field_names = {f.name for f in PUBLIC_FIELDS}
-        assert set(_DEFAULT_SEARCH_FIELDS) <= public_field_names

src/documents/tests/search/test_error_routing.py

@@ -1,247 +0,0 @@
-"""Diagnostics route by Cause, and user-facing messages are host-owned.
-
-whoosh-compat documents ``Diagnostic.message`` as developer output with no
-stability guarantee, so it must never reach an HTTP response body.
-"""
-
-from __future__ import annotations
-
-import logging
-from datetime import UTC
-
-import pytest
-import tantivy
-from whoosh_compat.errors import Diagnostic
-from whoosh_compat.errors import DiagnosticKind
-from whoosh_compat.errors import QueryError
-from whoosh_compat.errors import cause_for
-from whoosh_compat.fields import FieldKind
-from whoosh_compat.fields import FieldRef
-
-from documents.search._errors import SearchQueryError
-from documents.search._query import _map_emit_error
-from documents.search._query import _single_diagnostic_to_error
-from documents.search._query import parse_user_query
-from documents.search._schema import build_schema
-from documents.search._tokenizer import register_tokenizers
-
-pytestmark = pytest.mark.search
-
-_LIBRARY_PROSE = "INTERNAL LIBRARY WORDING WITH raw tantivy detail"
-
-
-@pytest.fixture(scope="module")
-def query_index() -> tantivy.Index:
-    """An in-memory, unstemmed index; these tests only parse, never index."""
-    idx = tantivy.Index(build_schema(), path=None)
-    register_tokenizers(idx, "")
-    return idx
-
-
-def _diagnostic(
-    kind: DiagnosticKind,
-    *,
-    field: FieldRef | None = FieldRef("title"),
-    field_kind: FieldKind | None = FieldKind.TEXT,
-) -> Diagnostic:
-    """A Diagnostic shaped like the emitter's, with the library's own
-    kind -> cause mapping rather than a hand-picked cause."""
-    return Diagnostic(
-        kind=kind,
-        cause=cause_for(kind),
-        message=_LIBRARY_PROSE,
-        field=field,
-        field_kind=field_kind,
-    )
-
-
-class TestEmitErrorRouting:
-    """Every Cause gets a distinguishable treatment, not just "a 400"."""
-
-    @pytest.mark.parametrize(
-        "kind",
-        [
-            DiagnosticKind.BACKEND_REJECTED,
-            DiagnosticKind.AST_INVALID_SHAPE,
-            DiagnosticKind.AST_UNKNOWN_FIELD,
-        ],
-    )
-    def test_internal_cause_is_not_converted(self, kind: DiagnosticKind) -> None:
-        """A library defect must surface as a 500 monitoring can see, not a
-        400 blaming the user."""
-        error = QueryError(_diagnostic(kind))
-        with pytest.raises(QueryError) as excinfo:
-            _map_emit_error(error)
-        assert excinfo.value is error
-
-    def test_misconfigured_cause_is_logged_and_becomes_a_400(
-        self,
-        caplog: pytest.LogCaptureFixture,
-    ) -> None:
-        kind = DiagnosticKind.SCHEMA_FIELD_MISSING
-        with caplog.at_level(logging.ERROR, logger="paperless.search"):
-            error = _map_emit_error(
-                QueryError(_diagnostic(kind, field=FieldRef("asn"))),
-            )
-        assert isinstance(error, SearchQueryError)
-        errors = [r for r in caplog.records if r.levelno == logging.ERROR]
-        assert len(errors) == 1
-        assert "asn" in errors[0].getMessage()
-        assert kind.name in errors[0].getMessage()
-
-    @pytest.mark.parametrize(
-        "kind",
-        [
-            DiagnosticKind.TEXT_RANGE,
-            DiagnosticKind.PATTERN_TOO_COMPLEX,
-            DiagnosticKind.EXISTS_REQUIRES_FAST,
-        ],
-    )
-    def test_unsupported_cause_is_a_400_with_no_operator_log(
-        self,
-        kind: DiagnosticKind,
-        caplog: pytest.LogCaptureFixture,
-    ) -> None:
-        """A query tantivy cannot run is the user's to fix; it must not page
-        an operator the way a registry/schema mismatch does.
-
-        EXISTS_REQUIRES_FAST is nominally MISCONFIGURED but belongs here: it
-        is decided from the registry's own FieldSpec, so it never reports a
-        disagreement anyone could resolve."""
-        with caplog.at_level(logging.WARNING, logger="paperless.search"):
-            error = _map_emit_error(QueryError(_diagnostic(kind)))
-        assert isinstance(error, SearchQueryError)
-        assert caplog.records == []
-
-    @pytest.mark.parametrize(
-        "kind",
-        [
-            DiagnosticKind.TEXT_RANGE,
-            DiagnosticKind.PATTERN_TOO_COMPLEX,
-            DiagnosticKind.EXISTS_REQUIRES_FAST,
-            DiagnosticKind.SCHEMA_FIELD_MISSING,
-        ],
-    )
-    def test_user_facing_message_never_echoes_library_prose(
-        self,
-        kind: DiagnosticKind,
-    ) -> None:
-        error = _map_emit_error(QueryError(_diagnostic(kind)))
-        assert _LIBRARY_PROSE not in str(error)
-
-    @pytest.mark.parametrize(
-        "kind",
-        [
-            DiagnosticKind.TEXT_RANGE,
-            DiagnosticKind.PATTERN_TOO_COMPLEX,
-            DiagnosticKind.EXISTS_REQUIRES_FAST,
-            DiagnosticKind.SCHEMA_FIELD_MISSING,
-        ],
-    )
-    def test_user_facing_message_names_the_field(
-        self,
-        kind: DiagnosticKind,
-    ) -> None:
-        """FieldRef.__str__ yields the canonical dotted name, including a
-        JSON subpath, so every user-reachable emit kind can name it."""
-        diagnostic = _diagnostic(
-            kind,
-            field=FieldRef("custom_fields", "value"),
-            field_kind=FieldKind.JSON,
-        )
-        error = _map_emit_error(QueryError(diagnostic))
-        assert "custom_fields.value" in str(error)
-
-
-class TestParseDiagnosticMessages:
-    """Parse-time diagnostics are host-worded too, off field_kind."""
-
-    def test_too_deep_is_a_400_without_library_prose(self) -> None:
-        error = _single_diagnostic_to_error(
-            _diagnostic(DiagnosticKind.TOO_DEEP, field=None, field_kind=None),
-        )
-        assert isinstance(error, SearchQueryError)
-        assert _LIBRARY_PROSE not in str(error)
-
-    @pytest.mark.parametrize(
-        ("kind", "field_kind"),
-        [
-            (DiagnosticKind.PATTERN_ON_NUMERIC, FieldKind.U64),
-            (DiagnosticKind.PATTERN_ON_BOOLEAN_EXISTS, FieldKind.BOOLEAN_EXISTS),
-            (DiagnosticKind.PATTERN_ON_SUBPATH, FieldKind.JSON),
-        ],
-    )
-    def test_pattern_on_kinds_name_the_field_and_its_kind(
-        self,
-        kind: DiagnosticKind,
-        field_kind: FieldKind,
-    ) -> None:
-        error = _single_diagnostic_to_error(
-            _diagnostic(kind, field=FieldRef("asn"), field_kind=field_kind),
-        )
-        message = str(error)
-        assert _LIBRARY_PROSE not in message
-        assert "asn" in message
-        assert field_kind.name.lower() in message
-
-    def test_single_char_bracket_range_names_the_field_and_the_value(self) -> None:
-        diagnostic = Diagnostic(
-            kind=DiagnosticKind.SINGLE_CHAR_BRACKET_RANGE,
-            cause=cause_for(DiagnosticKind.SINGLE_CHAR_BRACKET_RANGE),
-            message=_LIBRARY_PROSE,
-            field=FieldRef("title"),
-            field_kind=FieldKind.TEXT,
-            raw_value="200[1-9]",
-        )
-        error = _single_diagnostic_to_error(diagnostic)
-        message = str(error)
-        assert isinstance(error, SearchQueryError)
-        assert _LIBRARY_PROSE not in message
-        assert "title" in message
-        assert "200[1-9]" in message
-
-
-class TestRealQueriesRouteCorrectly:
-    """The routing table against diagnostics emit() really produces."""
-
-    def test_text_range_is_a_400_naming_the_field(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        with pytest.raises(SearchQueryError) as excinfo:
-            parse_user_query(query_index, "title:[a to b]", UTC)
-        assert "title" in str(excinfo.value)
-
-    def test_wildcard_on_a_numeric_field_is_a_400_naming_the_field(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        with pytest.raises(SearchQueryError) as excinfo:
-            parse_user_query(query_index, "asn:12*", UTC)
-        assert "asn" in str(excinfo.value)
-
-    def test_single_char_bracket_range_is_a_400_naming_field_and_value(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        with pytest.raises(SearchQueryError) as excinfo:
-            parse_user_query(query_index, "title:200[1-9]", UTC)
-        message = str(excinfo.value)
-        assert "title" in message
-        assert "200[1-9]" in message
-
-    def test_internal_diagnostic_escapes_as_a_query_error(
-        self,
-        query_index: tantivy.Index,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        """The one case with no query text that reaches it: emit() reporting
-        a defect in itself must not be converted to a user-facing 400."""
-        import documents.search._query as query_mod
-
-        def raise_internal(*args: object, **kwargs: object) -> None:
-            raise QueryError(_diagnostic(DiagnosticKind.BACKEND_REJECTED))
-
-        monkeypatch.setattr(query_mod, "tantivy_emit", raise_internal)
-        with pytest.raises(QueryError):
-            parse_user_query(query_index, "invoice", UTC)

src/documents/tests/search/test_exists_on_json_fields.py

@@ -1,92 +0,0 @@
-"""``field:*`` on a JSON field is user error, not an operator alert.
-
-whoosh-compat classifies EXISTS_REQUIRES_FAST as MISCONFIGURED, and
-_map_emit_error used to route every MISCONFIGURED diagnostic to an ERROR log.
-But the kind is decided from the registry's own FieldSpec (kind plus fast)
-without consulting the index schema, and field_descriptors() builds the JSON
-fields non-fast deliberately, so nothing is misconfigured and no operator
-action can clear the condition. Any authenticated user could otherwise emit
-ERROR lines in a loop by repeating ``notes:*``.
-
-SCHEMA_FIELD_MISSING, the other MISCONFIGURED kind, does compare the registry
-against the live schema, so it stays an ERROR.
-"""
-
-from __future__ import annotations
-
-import logging
-from datetime import UTC
-
-import pytest
-import tantivy
-from whoosh_compat.errors import Diagnostic
-from whoosh_compat.errors import DiagnosticKind
-from whoosh_compat.errors import QueryError
-from whoosh_compat.errors import cause_for
-from whoosh_compat.fields import FieldKind
-from whoosh_compat.fields import FieldRef
-
-from documents.search._errors import SearchQueryError
-from documents.search._query import _map_emit_error
-from documents.search._query import parse_user_query
-from documents.search._schema import build_schema
-from documents.search._tokenizer import register_tokenizers
-
-pytestmark = pytest.mark.search
-
-# Every spelling of "does this JSON field have a value" a user can type.
-EXISTS_QUERIES = [
-    "notes:*",
-    "notes.note:*",
-    "notes.user:*",
-    "custom_fields:*",
-    "custom_fields.name:*",
-    "custom_fields.value:*",
-]
-
-
-@pytest.fixture(scope="module")
-def query_index() -> tantivy.Index:
-    idx = tantivy.Index(build_schema(), path=None)
-    register_tokenizers(idx, "")
-    return idx
-
-
-class TestJsonExistsIsUserError:
-    @pytest.mark.parametrize("query", EXISTS_QUERIES)
-    def test_query_is_a_400_that_emits_no_error_log(
-        self,
-        query_index: tantivy.Index,
-        caplog: pytest.LogCaptureFixture,
-        query: str,
-    ) -> None:
-        with caplog.at_level(logging.WARNING, logger="paperless.search"):
-            with pytest.raises(SearchQueryError) as excinfo:
-                parse_user_query(query_index, query, UTC)
-        assert query.split(":", maxsplit=1)[0] in str(excinfo.value)
-        assert [r for r in caplog.records if r.levelno >= logging.ERROR] == []
-
-
-class TestGenuineMisconfigurationStillLogs:
-    def test_schema_field_missing_is_an_error_log(
-        self,
-        caplog: pytest.LogCaptureFixture,
-    ) -> None:
-        """The registry naming a field the index schema does not have is a
-        real mismatch an operator can fix, so it keeps the alert."""
-        kind = DiagnosticKind.SCHEMA_FIELD_MISSING
-        error = QueryError(
-            Diagnostic(
-                kind=kind,
-                cause=cause_for(kind),
-                message="field 'asn' is not defined in the index schema",
-                field=FieldRef("asn"),
-                field_kind=FieldKind.U64,
-            ),
-        )
-        with caplog.at_level(logging.ERROR, logger="paperless.search"):
-            mapped = _map_emit_error(error)
-        assert isinstance(mapped, SearchQueryError)
-        records = [r for r in caplog.records if r.levelno == logging.ERROR]
-        assert len(records) == 1
-        assert kind.name in records[0].getMessage()

src/documents/tests/search/test_fields.py

@@ -1,10 +0,0 @@
-from whoosh_compat import FieldKind
-
-from documents.search._fields import PUBLIC_FIELDS
-
-
-class TestPublicFields:
-    def test_json_fields_have_subpaths(self) -> None:
-        for field in PUBLIC_FIELDS:
-            if field.kind is FieldKind.JSON:
-                assert field.subpaths, f"{field.name} is JSON but has no subpaths"

src/documents/tests/search/test_fuzzy_tokenization.py

@@ -1,174 +0,0 @@
-"""The words the fuzzy blend clause hands back to tantivy's parser.
-
-The clause re-parses a word string through tantivy, which analyzes it
-again, so the words must be the query's raw text rather than the analyzed
-text (analysis is not idempotent), and must still be split into plain
-words so that hyphenated, dotted and quoted terms keep contributing.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from pytest_django.fixtures import SettingsWrapper
-
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-@pytest.fixture(autouse=True)
-def fuzzy_enabled(settings: SettingsWrapper) -> None:
-    """Enable the fuzzy blend clause. The threshold doubles as a minimum
-    score filter, so it is set to 0.0: every hit passes and the test sees
-    the clause's matching behaviour, not the filter's."""
-    settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.0
-
-
-class TestFuzzyClauseWords:
-    def test_a_stemmed_word_is_not_stemmed_a_second_time(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """'universities' stems to 'univers'; feeding that back to tantivy
-        stems it again to 'univ', whose fuzzy prefix reaches unrelated
-        words. The clause must stay wide enough for a typo and no wider."""
-        wanted = _index(
-            backend,
-            title="A",
-            content="universities of europe",
-            checksum="fuzz-stem-1",
-        )
-        typo = _index(
-            backend,
-            title="B",
-            content="universties of europe",
-            checksum="fuzz-stem-2",
-        )
-        _index(
-            backend,
-            title="C",
-            content="univalent chemical bonds",
-            checksum="fuzz-stem-3",
-        )
-        _index(
-            backend,
-            title="D",
-            content="unicycle repair manual",
-            checksum="fuzz-stem-4",
-        )
-
-        assert _matched_ids(backend, "universities") == {wanted.pk, typo.pk}
-
-    def test_a_hyphenated_term_still_reaches_the_clause(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """'COVID-19' is one raw token: unless it is split into words, it
-        carries characters the re-parse would read as grammar, is dropped,
-        and the whole query loses its fuzzy clause."""
-        misspelled = _index(
-            backend,
-            title="A",
-            content="covidx testing results",
-            checksum="fuzz-hyphen-1",
-        )
-
-        assert _matched_ids(backend, "COVID-19") == {misspelled.pk}
-
-    def test_a_phrase_still_reaches_the_clause(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """A phrase is one raw token carrying a space, and is the whole
-        query's only free text here."""
-        near_miss = _index(
-            backend,
-            title="A",
-            content="taxation reportage weekly",
-            checksum="fuzz-phrase-1",
-        )
-
-        assert _matched_ids(backend, '"tax reports"') == {near_miss.pk}
-
-
-class TestBooleanKeywordsInRawText:
-    """Tantivy's boolean keywords are word runs, so they survive the cut
-    into words and its own parser reads them as grammar. Raw query text
-    reaches that parser with its case intact, so a quoted phrase can carry
-    them in."""
-
-    @pytest.fixture
-    def corpus(self, backend: TantivyBackend) -> dict[str, int]:
-        both = _index(
-            backend,
-            title="A",
-            content="taxation reportage weekly",
-            checksum="fuzz-kw-1",
-        )
-        tax_only = _index(
-            backend,
-            title="B",
-            content="taxation only here",
-            checksum="fuzz-kw-2",
-        )
-        report_only = _index(
-            backend,
-            title="C",
-            content="reportage only here",
-            checksum="fuzz-kw-3",
-        )
-        return {
-            "both": both.pk,
-            "tax_only": tax_only.pk,
-            "report_only": report_only.pk,
-        }
-
-    @pytest.mark.parametrize(
-        "query",
-        [
-            pytest.param('"tax AND reports"', id="and"),
-            pytest.param('"tax OR reports"', id="or"),
-            pytest.param('"tax NOT reports"', id="not"),
-            pytest.param('"tax IN reports"', id="in"),
-        ],
-    )
-    def test_a_keyword_inside_a_phrase_stays_an_ordinary_word(
-        self,
-        backend: TantivyBackend,
-        corpus: dict[str, int],
-        query: str,
-    ) -> None:
-        """The phrase asks for three words, so the clause must stay the
-        disjunction it is for '"tax reports"': AND must not turn it into a
-        conjunction, NOT must not give it its own exclusion, IN must not
-        fail the parse."""
-        assert _matched_ids(backend, '"tax reports"') == set(corpus.values())
-        assert _matched_ids(backend, query) == set(corpus.values())
-
-    def test_a_trailing_keyword_does_not_drop_the_clause(
-        self,
-        backend: TantivyBackend,
-        corpus: dict[str, int],
-    ) -> None:
-        """'tax AND' is a syntax error to tantivy's parser, which would
-        cost the whole query its fuzzy clause."""
-        assert _matched_ids(backend, '"tax AND"') == {
-            corpus["both"],
-            corpus["tax_only"],
-        }

src/documents/tests/search/test_highlight_query_guard.py

@@ -1,192 +0,0 @@
-"""Regression coverage for the unguarded TEXT-mode highlight query.
-
-parse_simple_text_highlight_query re-parses simple-search tokens through
-Tantivy's query-string parser to build a SnippetGenerator-compatible query.
-Simple-search tokens keep arbitrary punctuation (quotes, colons, brackets,
-slashes), so any token carrying Tantivy query grammar raised an unguarded
-ValueError. The search itself had already succeeded by the time this ran:
-only the highlight step failed, and with the DocumentViewSet.list
-exception handler narrowed elsewhere on this branch, that ValueError now
-reaches the client as a bare 500 rather than a 400.
-
-Covers three angles:
-  - the query builder itself: quoting each token as its own escaped phrase
-    should let it parse instead of raising, for every failure mode a plain-
-    text query can trigger (syntax error, unknown field, unsupported regex).
-  - highlight_hits: even when a token still can't be expressed as a
-    highlight query, the guard must fall back to a query that still
-    produces usable highlight HTML, not silently empty ones.
-  - the real API endpoint: pinning the previously-500 status to 200.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-import tantivy
-from rest_framework import status
-
-from documents.search._backend import SearchMode
-from documents.search._query import parse_simple_text_highlight_query
-from documents.search._schema import build_schema
-from documents.search._tokenizer import register_tokenizers
-from documents.tests.factories import DocumentFactory
-
-if TYPE_CHECKING:
-    from rest_framework.test import APIClient
-
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-# Each spelling below trips a different Tantivy parser failure mode:
-#   'a"b'     -> Syntax Error (unterminated quote)
-#   foo:bar   -> unknown field
-#   (a        -> Syntax Error (unbalanced group)
-#   [a        -> Syntax Error (unbalanced range)
-#   /a/       -> Unsupported query (regex queries disallowed)
-_MALFORMED_QUERIES = [
-    pytest.param('a"b', id="unterminated_quote"),
-    pytest.param("foo:bar", id="unknown_field"),
-    pytest.param("(a", id="unbalanced_group"),
-    pytest.param("[a", id="unbalanced_range"),
-    pytest.param("/a/", id="unsupported_regex"),
-]
-
-
-@pytest.fixture(scope="module")
-def query_index() -> tantivy.Index:
-    """An in-memory, unstemmed index for parse-only tests."""
-    schema = build_schema()
-    idx = tantivy.Index(schema, path=None)
-    register_tokenizers(idx, "")
-    return idx
-
-
-class TestParseSimpleTextHighlightQueryDoesNotRaise:
-    """The query builder itself must tolerate Tantivy syntax in its tokens."""
-
-    @pytest.mark.parametrize("raw_query", _MALFORMED_QUERIES)
-    def test_malformed_token_does_not_raise(
-        self,
-        query_index: tantivy.Index,
-        raw_query: str,
-    ) -> None:
-        assert isinstance(
-            parse_simple_text_highlight_query(query_index, raw_query),
-            tantivy.Query,
-        )
-
-
-class TestHighlightHitsProducesUsableHighlights:
-    """highlight_hits must keep producing real <b>-wrapped snippet HTML for
-    these queries, not merely avoid raising."""
-
-    @pytest.mark.parametrize(
-        "raw_query",
-        [*_MALFORMED_QUERIES, pytest.param("plain text", id="plain_text_sanity")],
-    )
-    def test_highlight_still_contains_matched_text(
-        self,
-        backend: TantivyBackend,
-        raw_query: str,
-    ) -> None:
-        doc = DocumentFactory.create(
-            title="probe",
-            content=f"needle content containing {raw_query} literally here",
-        )
-        backend.add_or_update(doc)
-
-        hits = backend.highlight_hits(
-            raw_query,
-            [doc.pk],
-            search_mode=SearchMode.TEXT,
-        )
-
-        assert len(hits) == 1
-        highlights = hits[0]["highlights"]
-        assert "content" in highlights, (
-            f"Expected a content highlight for {raw_query!r}, got: {highlights!r}"
-        )
-        assert "<b>" in highlights["content"], (
-            f"Highlight for {raw_query!r} carries no matched-term markup: "
-            f"{highlights['content']!r}"
-        )
-
-
-class TestHighlightGuardDiscriminatesOnValueError:
-    """The guard added to highlight_hits must catch exactly ValueError, the
-    same shape as the sibling notes_text guard, and let anything else
-    through -- so a real library defect is never mistaken for a harmless
-    syntax error."""
-
-    def test_non_value_error_is_not_swallowed(
-        self,
-        backend: TantivyBackend,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_runtime_error(*args: object, **kwargs: object) -> object:
-            raise RuntimeError("synthetic bug, unrelated to query syntax")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_simple_text_highlight_query",
-            raise_runtime_error,
-        )
-
-        doc = DocumentFactory.create(title="probe", content="anything here")
-        backend.add_or_update(doc)
-
-        with pytest.raises(RuntimeError):
-            backend.highlight_hits(
-                "anything",
-                [doc.pk],
-                search_mode=SearchMode.TEXT,
-            )
-
-
-@pytest.mark.usefixtures("_search_index")
-class TestApiNoLongerReturns500:
-    """Pins the actual regression: a matching TEXT-mode search whose query
-    string carries Tantivy syntax must return results, not a server error."""
-
-    @pytest.mark.parametrize("raw_query", _MALFORMED_QUERIES)
-    def test_malformed_text_query_returns_200(
-        self,
-        admin_client: APIClient,
-        raw_query: str,
-    ) -> None:
-        from documents.search import get_backend
-
-        doc = DocumentFactory.create(
-            title="probe",
-            content=f"needle content containing {raw_query} literally here",
-        )
-        get_backend().add_or_update(doc)
-
-        response = admin_client.get(f"/api/documents/?text={raw_query}")
-
-        assert response.status_code == status.HTTP_200_OK
-        assert response.data["count"] == 1
-
-    def test_plain_text_query_still_returns_200(
-        self,
-        admin_client: APIClient,
-    ) -> None:
-        """Sanity check: the guard must not mask a total failure of the
-        ordinary highlight path."""
-        from documents.search import get_backend
-
-        doc = DocumentFactory.create(
-            title="probe",
-            content="needle content containing plain text literally here",
-        )
-        get_backend().add_or_update(doc)
-
-        response = admin_client.get("/api/documents/?text=plain text")
-
-        assert response.status_code == status.HTTP_200_OK
-        assert response.data["count"] == 1

src/documents/tests/search/test_json_field_prefixes.py

@@ -1,148 +0,0 @@
-"""Bare notes:/custom_fields: prefix resolution.
-
-"notes:foo"/"custom_fields:foo" were valid fielded searches before the
-whoosh-compat migration. The registry only exposes them as JSON subpaths, so
-each JSON FieldSpec declares a default subpath (SubpathSpec(default=True)):
-notes: resolves to notes.note:, custom_fields: resolves to
-custom_fields.value:. This replaced an earlier regex-based rewrite
-(_rewrite_bare_json_field_prefixes) that ran on the raw query string before
-parsing and was blind to quoting, so a phrase like
-content:"payment notes: none" was silently corrupted into a notes-field
-search and matched nothing. Resolving the default subpath inside the parser
-instead means quoting is already understood by the time it happens.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-from django.contrib.auth.models import User
-
-from documents.models import CustomField
-from documents.models import CustomFieldInstance
-from documents.models import Document
-from documents.models import Note
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-class TestBareJsonFieldPrefixes:
-    def test_bare_notes_prefix_searches_note_text(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        alice = User.objects.create_user(username="alice")
-        with_note = Document.objects.create(
-            title="Has note",
-            content="x",
-            checksum="bare-notes-with",
-        )
-        Note.objects.create(document=with_note, user=alice, note="crocodile")
-        backend.add_or_update(with_note)
-        # This document's CONTENT contains the words a demoted text search
-        # would match; it must NOT match once the prefix addresses notes.
-        _index(
-            backend,
-            title="Notes about things",
-            content="notes crocodile mention",
-            checksum="bare-notes-decoy",
-        )
-        assert _matched_ids(backend, "notes:crocodile") == {with_note.pk}
-
-    def test_bare_custom_fields_prefix_searches_values(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        field = CustomField.objects.create(
-            name="Policy Number",
-            data_type=CustomField.FieldDataType.STRING,
-        )
-        with_value = Document.objects.create(
-            title="Has field",
-            content="x",
-            checksum="bare-cf-with",
-        )
-        CustomFieldInstance.objects.create(
-            document=with_value,
-            field=field,
-            value_text="crocodile",
-        )
-        backend.add_or_update(with_value)
-        _index(
-            backend,
-            title="Custom things",
-            content="custom fields crocodile",
-            checksum="bare-cf-decoy",
-        )
-        assert _matched_ids(backend, "custom_fields:crocodile") == {with_value.pk}
-
-    def test_subpath_spellings_are_untouched(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        bob = User.objects.create_user(username="bob")
-        doc = Document.objects.create(
-            title="Bob note",
-            content="x",
-            checksum="bare-subpath",
-        )
-        Note.objects.create(document=doc, user=bob, note="remark")
-        backend.add_or_update(doc)
-        assert _matched_ids(backend, "notes.user:bob") == {doc.pk}
-        assert _matched_ids(backend, "notes.note:remark") == {doc.pk}
-
-
-class TestQuotedPhraseContainingNotesColonIsNotCorrupted:
-    """The regex rewrite this migration removes was blind to quoting: it
-    matched "notes:" anywhere in the raw query string, including inside an
-    already-quoted phrase on an unrelated field, silently turning
-    content:"payment notes: none" into a notes-field search that matched
-    nothing. Resolving the default subpath during parsing (which is
-    quote-aware) fixes this."""
-
-    def test_quoted_phrase_with_notes_colon_matches_by_content(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        target = _index(
-            backend,
-            title="Statement",
-            content="payment notes: none",
-            checksum="quoted-phrase-notes-colon",
-        )
-        assert _matched_ids(
-            backend,
-            'content:"payment notes: none"',
-        ) == {target.pk}
-
-    def test_quoted_phrase_matches_the_same_document_unquoted(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        # Same document, phrasing without the colon: this proves the fix is
-        # about quote-awareness, not about the words themselves being
-        # unsearchable.
-        target = _index(
-            backend,
-            title="Statement",
-            content="payment notes none",
-            checksum="quoted-phrase-no-colon",
-        )
-        assert _matched_ids(
-            backend,
-            'content:"payment notes none"',
-        ) == {target.pk}

src/documents/tests/search/test_json_subpath_completeness.py

@@ -1,83 +0,0 @@
-"""Every declared JSON subpath must actually be written to the index.
-
-PUBLIC_FIELDS declares each JSON field's subpaths (e.g. ``notes`` ->
-{"user", "note"}), but nothing coupled that declaration to what
-``_backend.py``'s document builder actually writes into the JSON blob at
-index time. A subpath declared but never written would be
-queryable-but-always-empty -- syntactically valid, silently matching
-nothing -- with no test failure anywhere.
-
-This indexes one real document carrying values for every JSON field
-(a Note, a CustomFieldInstance) and inspects the document's own stored
-JSON payload, rather than running field-specific queries: that way a
-future JSON field's subpaths are covered automatically, without a new
-per-subpath query having to be added by hand each time.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-import tantivy
-from django.contrib.auth.models import User
-from whoosh_compat import FieldKind
-
-from documents.models import CustomField
-from documents.models import CustomFieldInstance
-from documents.models import Document
-from documents.models import Note
-from documents.search._fields import PUBLIC_FIELDS
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-class TestJsonSubpathsAreWrittenAtIndexTime:
-    def test_every_declared_json_subpath_appears_in_the_stored_document(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        user = User.objects.create_user(username="completeness-user")
-        field = CustomField.objects.create(
-            name="Completeness Field",
-            data_type=CustomField.FieldDataType.STRING,
-        )
-        doc = Document.objects.create(
-            title="Completeness doc",
-            content="x",
-            checksum="json-subpath-completeness",
-        )
-        Note.objects.create(document=doc, user=user, note="a note")
-        CustomFieldInstance.objects.create(
-            document=doc,
-            field=field,
-            value_text="a value",
-        )
-        backend.add_or_update(doc)
-
-        index = backend._index
-        searcher = index.searcher()
-        hits = searcher.search(
-            tantivy.Query.term_query(index.schema, "id", doc.pk),
-            limit=1,
-        ).hits
-        assert hits, "the document was not indexed"
-        stored = searcher.doc(hits[0][1]).to_dict()
-
-        json_fields = [f for f in PUBLIC_FIELDS if f.kind is FieldKind.JSON]
-        assert json_fields, "no JSON fields declared - fixture is stale"
-        for field_spec in json_fields:
-            stored_values = stored.get(field_spec.name)
-            assert stored_values, (
-                f"{field_spec.name} was not written to the index at all"
-            )
-            written_keys = stored_values[0].keys()
-            for subpath in field_spec.subpaths:
-                assert subpath in written_keys, (
-                    f"{field_spec.name}.{subpath} is declared in PUBLIC_FIELDS "
-                    "but _backend.py's document builder never writes it - it "
-                    "would be queryable but always empty"
-                )

src/documents/tests/search/test_keyword_pattern_literal.py

@@ -1,60 +0,0 @@
-"""Wildcard patterns on KEYWORD fields must stay literal.
-
-``checksum`` is the only KEYWORD field: it is indexed with the raw tokenizer,
-so its terms are never lowercased, folded or stemmed. Running its wildcard
-patterns through the stemming normalizer rewrote hex prefixes ("ceded" ->
-"cede") and returned documents whose checksum did not start with what the user
-typed, which for an identity field is a wrong answer.
-
-This covers only the registry-level normalizer, which is all that exists to
-prove at this point in the stack: user queries are not yet routed through
-whoosh-compat (that lands with the query-layer PR), so the same fact proven
-end to end against real indexed documents lives in
-``test_checksum_prefix_queries.py``.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.search._registry import get_field_registry
-
-if TYPE_CHECKING:
-    from whoosh_compat import FieldRegistry
-    from whoosh_compat import PatternNormalizer
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _normalizer(registry: FieldRegistry, name: str) -> PatternNormalizer:
-    ref = registry.make_ref(name)
-    assert ref is not None
-    resolved = registry.resolve(ref)
-    assert resolved is not None
-    assert resolved.spec.pattern_normalizer is not None
-    return resolved.spec.pattern_normalizer
-
-
-class TestKeywordPatternNormalizer:
-    @pytest.mark.parametrize(
-        "run",
-        [
-            pytest.param("ceded", id="stems_to_cede"),
-            pytest.param("added", id="stems_to_ad"),
-            pytest.param("cafed", id="stems_to_cafe"),
-        ],
-    )
-    def test_keyword_runs_are_folded_not_stemmed(self, run: str) -> None:
-        """One form, the run as typed: a KEYWORD pattern must never be widened
-        to a stem, which would return checksums that do not start with what
-        the user typed."""
-        normalize = _normalizer(get_field_registry("en"), "checksum")
-        assert normalize(run) == run
-
-    def test_text_runs_still_offer_their_stem(self) -> None:
-        """A TEXT field offers the stem alongside the typed run, so a term
-        matching either one is reachable."""
-        normalize = _normalizer(get_field_registry("en"), "title")
-        assert tuple(normalize("Running")) == ("running", "run")

src/documents/tests/search/test_permission_field_isolation.py

@@ -1,148 +0,0 @@
-"""Permission filtering must hold against the real indexed document shape.
-
-Only three of the index's unsigned ``*_id`` columns are load-bearing:
-``owner_id``, ``viewer_id`` and ``viewer_group_id``, all read by
-build_permission_filter. The rest (correspondent/document_type/storage_path/tag
-ids) were written on every document and read by nothing, and were dropped.
-
-These tests index real Documents through the backend's own document builder and
-assert result-level visibility per user, so a mistake about which columns are
-load-bearing shows up as documents leaking across users rather than as a passing
-unit test over a hand-built index.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-from django.contrib.auth.models import Group
-from django.contrib.auth.models import User
-from guardian.shortcuts import assign_perm
-
-from documents.models import Correspondent
-from documents.models import Document
-from documents.models import DocumentType
-from documents.models import StoragePath
-from documents.models import Tag
-
-if TYPE_CHECKING:
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-@pytest.fixture
-def owner() -> User:
-    return User.objects.create_user(username="owner")
-
-
-@pytest.fixture
-def stranger() -> User:
-    return User.objects.create_user(username="stranger")
-
-
-@pytest.fixture
-def viewer() -> User:
-    return User.objects.create_user(username="viewer")
-
-
-@pytest.fixture
-def group_member() -> User:
-    user = User.objects.create_user(username="group_member")
-    user.groups.add(Group.objects.create(name="accounting"))
-    return user
-
-
-class TestPermissionFilteringOnIndexedDocuments:
-    def test_unowned_document_is_visible_to_everyone(
-        self,
-        backend: TantivyBackend,
-        stranger: User,
-    ) -> None:
-        doc = Document.objects.create(
-            title="Public Invoice",
-            content="invoice total due",
-            checksum="perm-unowned",
-        )
-        backend.add_or_update(doc)
-
-        assert backend.search_ids("invoice", user=stranger) == [doc.pk]
-
-    def test_owned_document_is_visible_only_to_its_owner(
-        self,
-        backend: TantivyBackend,
-        owner: User,
-        stranger: User,
-    ) -> None:
-        doc = Document.objects.create(
-            title="Private Invoice",
-            content="invoice total due",
-            checksum="perm-owned",
-            owner=owner,
-        )
-        backend.add_or_update(doc)
-
-        assert backend.search_ids("invoice", user=owner) == [doc.pk]
-        assert backend.search_ids("invoice", user=stranger) == []
-
-    def test_explicitly_shared_document_is_visible_to_the_viewer(
-        self,
-        backend: TantivyBackend,
-        owner: User,
-        viewer: User,
-        stranger: User,
-    ) -> None:
-        doc = Document.objects.create(
-            title="Shared Invoice",
-            content="invoice total due",
-            checksum="perm-shared-user",
-            owner=owner,
-        )
-        assign_perm("view_document", viewer, doc)
-        backend.add_or_update(doc)
-
-        assert backend.search_ids("invoice", user=viewer) == [doc.pk]
-        assert backend.search_ids("invoice", user=stranger) == []
-
-    def test_group_shared_document_is_visible_to_group_members(
-        self,
-        backend: TantivyBackend,
-        owner: User,
-        group_member: User,
-        stranger: User,
-    ) -> None:
-        doc = Document.objects.create(
-            title="Group Invoice",
-            content="invoice total due",
-            checksum="perm-shared-group",
-            owner=owner,
-        )
-        assign_perm("view_document", group_member.groups.first(), doc)
-        backend.add_or_update(doc)
-
-        assert backend.search_ids("invoice", user=group_member) == [doc.pk]
-        assert backend.search_ids("invoice", user=stranger) == []
-
-    def test_metadata_does_not_widen_visibility(
-        self,
-        backend: TantivyBackend,
-        owner: User,
-        stranger: User,
-    ) -> None:
-        """A document carrying correspondent/type/storage-path/tag metadata is
-        still filtered by owner alone."""
-        doc = Document.objects.create(
-            title="Tagged Invoice",
-            content="invoice total due",
-            checksum="perm-metadata",
-            owner=owner,
-            correspondent=Correspondent.objects.create(name="ACME"),
-            document_type=DocumentType.objects.create(name="Bill"),
-            storage_path=StoragePath.objects.create(name="Archive", path="archive/"),
-        )
-        doc.tags.add(Tag.objects.create(name="paid"))
-        backend.add_or_update(doc)
-
-        assert backend.search_ids("invoice", user=owner) == [doc.pk]
-        assert backend.search_ids("invoice", user=stranger) == []

src/documents/tests/search/test_query.py

@@ -1,96 +1,448 @@
 from __future__ import annotations
 
+import re
 from datetime import UTC
 from datetime import datetime
+from datetime import tzinfo
 from typing import TYPE_CHECKING
+from zoneinfo import ZoneInfo
 
 import pytest
 import tantivy
 import time_machine
 
-from documents.search._backend import build_permission_filter
-from documents.search._errors import InvalidDateQuery
-from documents.search._errors import InvalidNumberQuery
-from documents.search._errors import MultipleSearchQueryErrors
-from documents.search._errors import SearchQueryError
+from documents.search._dates import _date_only_range
+from documents.search._dates import _datetime_range
+from documents.search._query import build_permission_filter
 from documents.search._query import parse_simple_text_highlight_query
 from documents.search._query import parse_user_query
 from documents.search._schema import build_schema
 from documents.search._tokenizer import register_tokenizers
+from documents.search._translate import InvalidDateQuery
+from documents.search._translate import translate_query
 
 if TYPE_CHECKING:
     from django.contrib.auth.base_user import AbstractBaseUser
 
 pytestmark = pytest.mark.search
 
-
-@pytest.fixture(scope="module")
-def query_index() -> tantivy.Index:
-    """An in-memory, unstemmed index shared read-only across this module's
-    parse-only tests (none of them index documents)."""
-    schema = build_schema()
-    idx = tantivy.Index(schema, path=None)
-    register_tokenizers(idx, "")
-    return idx
+EASTERN = ZoneInfo("America/New_York")  # UTC-5 / UTC-4 (DST)
+AUCKLAND = ZoneInfo("Pacific/Auckland")  # UTC+13 in southern-hemisphere summer
 
 
-@pytest.fixture(scope="module")
-def populated_index() -> tantivy.Index:
-    """An index holding one document, so a query matching nothing is
-    distinguishable from one matching everything."""
-    idx = tantivy.Index(build_schema(), path=None)
-    register_tokenizers(idx, "")
-    writer = idx.writer()
-    doc = tantivy.Document()
-    doc.add_unsigned("id", 1)
-    doc.add_text("content", "needle in indexed content")
-    writer.add_document(doc)
-    writer.commit()
-    idx.reload()
-    return idx
+def _range(result: str, field: str) -> tuple[str, str]:
+    # Half-open period ranges close with "}" (exclusive); exact-instant ranges
+    # (full ISO datetimes, "now", relative offsets) close with "]" (inclusive).
+    m = re.search(rf"{field}:\[(.+?) TO (.+?)[\]}}]", result)
+    assert m, f"No range for {field!r} in: {result!r}"
+    return m.group(1), m.group(2)
 
 
-def _highlight_hit_count(index: tantivy.Index, raw_query: str) -> int:
-    query = parse_simple_text_highlight_query(index, raw_query)
-    return index.searcher().search(query, limit=1).count
+class TestCreatedDateField:
+    """
+    created is a Django DateField: indexed as midnight UTC of the local calendar
+    date. No offset arithmetic needed - the local calendar date is what matters.
+    """
+
+    @pytest.mark.parametrize(
+        ("tz", "expected_lo", "expected_hi"),
+        [
+            pytest.param(UTC, "2026-03-28T00:00:00Z", "2026-03-29T00:00:00Z", id="utc"),
+            pytest.param(
+                EASTERN,
+                "2026-03-28T00:00:00Z",
+                "2026-03-29T00:00:00Z",
+                id="eastern_same_calendar_date",
+            ),
+        ],
+    )
+    @time_machine.travel(datetime(2026, 3, 28, 15, 30, tzinfo=UTC), tick=False)
+    def test_today(self, tz: tzinfo, expected_lo: str, expected_hi: str) -> None:
+        lo, hi = _range(translate_query("created:today", tz), "created")
+        assert lo == expected_lo
+        assert hi == expected_hi
+
+    @time_machine.travel(datetime(2026, 3, 28, 3, 0, tzinfo=UTC), tick=False)
+    def test_today_auckland_ahead_of_utc(self) -> None:
+        # UTC 03:00 -> Auckland (UTC+13) = 16:00 same date; local date = 2026-03-28
+        lo, _ = _range(
+            translate_query("created:today", AUCKLAND),
+            "created",
+        )
+        assert lo == "2026-03-28T00:00:00Z"
+
+    @pytest.mark.parametrize(
+        ("field", "keyword", "expected_lo", "expected_hi"),
+        [
+            pytest.param(
+                "created",
+                "yesterday",
+                "2026-03-27T00:00:00Z",
+                "2026-03-28T00:00:00Z",
+                id="yesterday",
+            ),
+            pytest.param(
+                "created",
+                "previous week",
+                "2026-03-16T00:00:00Z",
+                "2026-03-23T00:00:00Z",
+                id="previous_week",
+            ),
+            pytest.param(
+                "created",
+                "this month",
+                "2026-03-01T00:00:00Z",
+                "2026-04-01T00:00:00Z",
+                id="this_month",
+            ),
+            pytest.param(
+                "created",
+                "previous month",
+                "2026-02-01T00:00:00Z",
+                "2026-03-01T00:00:00Z",
+                id="previous_month",
+            ),
+            pytest.param(
+                "created",
+                "this year",
+                "2026-01-01T00:00:00Z",
+                "2027-01-01T00:00:00Z",
+                id="this_year",
+            ),
+            pytest.param(
+                "created",
+                "previous year",
+                "2025-01-01T00:00:00Z",
+                "2026-01-01T00:00:00Z",
+                id="previous_year",
+            ),
+        ],
+    )
+    @time_machine.travel(datetime(2026, 3, 28, 15, 0, tzinfo=UTC), tick=False)
+    def test_date_keywords(
+        self,
+        field: str,
+        keyword: str,
+        expected_lo: str,
+        expected_hi: str,
+    ) -> None:
+        # 2026-03-28 is Saturday; Mon-Sun week calculation built into expectations
+        query = f"{field}:{keyword}"
+        lo, hi = _range(translate_query(query, UTC), field)
+        assert lo == expected_lo
+        assert hi == expected_hi
+
+    @time_machine.travel(datetime(2026, 12, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_this_month_december_wraps_to_next_year(self) -> None:
+        # December: next month must roll over to January 1 of next year
+        lo, hi = _range(
+            translate_query("created:this month", UTC),
+            "created",
+        )
+        assert lo == "2026-12-01T00:00:00Z"
+        assert hi == "2027-01-01T00:00:00Z"
+
+    @time_machine.travel(datetime(2026, 1, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_last_month_january_wraps_to_previous_year(self) -> None:
+        # January: last month must roll back to December 1 of previous year
+        lo, hi = _range(
+            translate_query("created:previous month", UTC),
+            "created",
+        )
+        assert lo == "2025-12-01T00:00:00Z"
+        assert hi == "2026-01-01T00:00:00Z"
+
+    @time_machine.travel(datetime(2026, 7, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_previous_quarter(self) -> None:
+        lo, hi = _range(
+            translate_query('created:"previous quarter"', UTC),
+            "created",
+        )
+        assert lo == "2026-04-01T00:00:00Z"
+        assert hi == "2026-07-01T00:00:00Z"
+
+    def test_unknown_keyword_raises(self) -> None:
+        with pytest.raises(ValueError, match="Unknown keyword"):
+            _date_only_range("bogus_keyword", UTC)
+
+
+class TestDateTimeFields:
+    """
+    added/modified store full UTC datetimes. Natural keywords must convert
+    the local day boundaries to UTC - timezone offset arithmetic IS required.
+    """
+
+    @time_machine.travel(datetime(2026, 3, 28, 15, 30, tzinfo=UTC), tick=False)
+    def test_added_today_eastern(self) -> None:
+        # EDT = UTC-4; local midnight 2026-03-28 00:00 EDT = 2026-03-28 04:00 UTC
+        lo, hi = _range(translate_query("added:today", EASTERN), "added")
+        assert lo == "2026-03-28T04:00:00Z"
+        assert hi == "2026-03-29T04:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 29, 2, 0, tzinfo=UTC), tick=False)
+    def test_added_today_auckland_midnight_crossing(self) -> None:
+        # UTC 02:00 on 2026-03-29 -> Auckland (UTC+13) = 2026-03-29 15:00 local
+        # Auckland midnight = UTC 2026-03-28 11:00
+        lo, hi = _range(translate_query("added:today", AUCKLAND), "added")
+        assert lo == "2026-03-28T11:00:00Z"
+        assert hi == "2026-03-29T11:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 15, 0, tzinfo=UTC), tick=False)
+    def test_modified_today_utc(self) -> None:
+        lo, hi = _range(
+            translate_query("modified:today", UTC),
+            "modified",
+        )
+        assert lo == "2026-03-28T00:00:00Z"
+        assert hi == "2026-03-29T00:00:00Z"
+
+    @pytest.mark.parametrize(
+        ("keyword", "expected_lo", "expected_hi"),
+        [
+            pytest.param(
+                "yesterday",
+                "2026-03-27T00:00:00Z",
+                "2026-03-28T00:00:00Z",
+                id="yesterday",
+            ),
+            pytest.param(
+                "previous week",
+                "2026-03-16T00:00:00Z",
+                "2026-03-23T00:00:00Z",
+                id="previous_week",
+            ),
+            pytest.param(
+                "this month",
+                "2026-03-01T00:00:00Z",
+                "2026-04-01T00:00:00Z",
+                id="this_month",
+            ),
+            pytest.param(
+                "previous month",
+                "2026-02-01T00:00:00Z",
+                "2026-03-01T00:00:00Z",
+                id="previous_month",
+            ),
+            pytest.param(
+                "this year",
+                "2026-01-01T00:00:00Z",
+                "2027-01-01T00:00:00Z",
+                id="this_year",
+            ),
+            pytest.param(
+                "previous year",
+                "2025-01-01T00:00:00Z",
+                "2026-01-01T00:00:00Z",
+                id="previous_year",
+            ),
+        ],
+    )
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_datetime_keywords_utc(
+        self,
+        keyword: str,
+        expected_lo: str,
+        expected_hi: str,
+    ) -> None:
+        # 2026-03-28 is Saturday; weekday()==5 so Monday=2026-03-23
+        lo, hi = _range(translate_query(f"added:{keyword}", UTC), "added")
+        assert lo == expected_lo
+        assert hi == expected_hi
+
+    @time_machine.travel(datetime(2026, 12, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_this_month_december_wraps_to_next_year(self) -> None:
+        # December: next month wraps to January of next year
+        lo, hi = _range(translate_query("added:this month", UTC), "added")
+        assert lo == "2026-12-01T00:00:00Z"
+        assert hi == "2027-01-01T00:00:00Z"
+
+    @time_machine.travel(datetime(2026, 1, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_last_month_january_wraps_to_previous_year(self) -> None:
+        # January: last month wraps back to December of previous year
+        lo, hi = _range(
+            translate_query("added:previous month", UTC),
+            "added",
+        )
+        assert lo == "2025-12-01T00:00:00Z"
+        assert hi == "2026-01-01T00:00:00Z"
+
+    @pytest.mark.parametrize(
+        ("query", "expected_lo", "expected_hi"),
+        [
+            pytest.param(
+                'added:"previous quarter"',
+                "2026-04-01T00:00:00Z",
+                "2026-07-01T00:00:00Z",
+                id="quoted_previous_quarter",
+            ),
+            pytest.param(
+                "added:previous month",
+                "2026-06-01T00:00:00Z",
+                "2026-07-01T00:00:00Z",
+                id="bare_previous_month",
+            ),
+            pytest.param(
+                "added:this month",
+                "2026-07-01T00:00:00Z",
+                "2026-08-01T00:00:00Z",
+                id="bare_this_month",
+            ),
+        ],
+    )
+    @time_machine.travel(datetime(2026, 7, 15, 12, 0, tzinfo=UTC), tick=False)
+    def test_legacy_natural_language_aliases(
+        self,
+        query: str,
+        expected_lo: str,
+        expected_hi: str,
+    ) -> None:
+        lo, hi = _range(translate_query(query, UTC), "added")
+        assert lo == expected_lo
+        assert hi == expected_hi
+
+    def test_unknown_keyword_raises(self) -> None:
+        with pytest.raises(ValueError, match="Unknown keyword"):
+            _datetime_range("bogus_keyword", UTC)
+
+
+class TestWhooshQueryRewriting:
+    """All Whoosh query syntax variants must be rewritten to ISO 8601 before Tantivy parses them."""
+
+    @time_machine.travel(datetime(2026, 3, 28, 15, 0, tzinfo=UTC), tick=False)
+    def test_compact_date_shim_rewrites_to_iso(self) -> None:
+        result = translate_query("created:20240115120000", UTC)
+        assert "2024-01-15" in result
+        assert "20240115120000" not in result
+
+    @time_machine.travel(datetime(2026, 3, 28, 15, 0, tzinfo=UTC), tick=False)
+    def test_relative_range_shim_removes_now(self) -> None:
+        result = translate_query("added:[now-7d TO now]", UTC)
+        assert "now" not in result
+        assert "2026-03-" in result
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_minus_7_days(self) -> None:
+        lo, hi = _range(
+            translate_query("added:[-7 days to now]", UTC),
+            "added",
+        )
+        assert lo == "2026-03-21T12:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_minus_1_week(self) -> None:
+        lo, hi = _range(
+            translate_query("added:[-1 week to now]", UTC),
+            "added",
+        )
+        assert lo == "2026-03-21T12:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_minus_1_month_uses_relativedelta(self) -> None:
+        # relativedelta(months=1) from 2026-03-28 = 2026-02-28 (not 29)
+        lo, hi = _range(
+            translate_query("created:[-1 month to now]", UTC),
+            "created",
+        )
+        assert lo == "2026-02-28T12:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_minus_1_year(self) -> None:
+        lo, hi = _range(
+            translate_query("modified:[-1 year to now]", UTC),
+            "modified",
+        )
+        assert lo == "2025-03-28T12:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_plural_unit_hours(self) -> None:
+        lo, hi = _range(
+            translate_query("added:[-3 hours to now]", UTC),
+            "added",
+        )
+        assert lo == "2026-03-28T09:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_bracket_case_insensitive(self) -> None:
+        result = translate_query("added:[-1 WEEK TO NOW]", UTC)
+        assert "now" not in result.lower()
+        lo, hi = _range(result, "added")
+        assert lo == "2026-03-21T12:00:00Z"
+        assert hi == "2026-03-28T12:00:00Z"
+
+    @time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False)
+    def test_relative_range_swaps_bounds_when_lo_exceeds_hi(self) -> None:
+        # [now+1h TO now-1h] has lo > hi before substitution; they must be swapped
+        lo, hi = _range(
+            translate_query("added:[now+1h TO now-1h]", UTC),
+            "added",
+        )
+        assert lo == "2026-03-28T11:00:00Z"
+        assert hi == "2026-03-28T13:00:00Z"
+
+    def test_8digit_created_date_field_always_uses_utc_midnight(self) -> None:
+        # created is a DateField: boundaries are always UTC midnight, no TZ offset
+        result = translate_query("created:20231201", EASTERN)
+        lo, hi = _range(result, "created")
+        assert lo == "2023-12-01T00:00:00Z"
+        assert hi == "2023-12-02T00:00:00Z"
+
+    def test_8digit_added_datetime_field_converts_local_midnight_to_utc(self) -> None:
+        # added is DateTimeField: midnight Dec 1 Eastern (EST = UTC-5) = 05:00 UTC
+        result = translate_query("added:20231201", EASTERN)
+        lo, hi = _range(result, "added")
+        assert lo == "2023-12-01T05:00:00Z"
+        assert hi == "2023-12-02T05:00:00Z"
+
+    def test_8digit_modified_datetime_field_converts_local_midnight_to_utc(
+        self,
+    ) -> None:
+        result = translate_query("modified:20231201", EASTERN)
+        lo, hi = _range(result, "modified")
+        assert lo == "2023-12-01T05:00:00Z"
+        assert hi == "2023-12-02T05:00:00Z"
+
+    def test_8digit_invalid_date_raises(self) -> None:
+        # The translation pipeline raises InvalidDateQuery for unparsable dates
+        # (e.g. month=13) so the API can surface a 400 telling the user the date
+        # is malformed instead of silently returning zero results.
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_query("added:20231340", UTC)
+        assert exc_info.value.field == "added"
+        assert exc_info.value.value == "20231340"
 
 
 class TestParseUserQuery:
     """parse_user_query runs the full preprocessing pipeline."""
 
+    @pytest.fixture
+    def query_index(self) -> tantivy.Index:
+        schema = build_schema()
+        idx = tantivy.Index(schema, path=None)
+        register_tokenizers(idx, "")
+        return idx
+
     def test_returns_tantivy_query(self, query_index: tantivy.Index) -> None:
         assert isinstance(parse_user_query(query_index, "invoice", UTC), tantivy.Query)
 
-    @pytest.mark.parametrize(
-        "raw_query",
-        [
-            pytest.param("invoice", id="plain_text"),
-            pytest.param("created:today", id="date_keyword"),
-            pytest.param("created:[2005 to 2009]", id="whoosh_date_range"),
-            pytest.param('added:"previous month"', id="quoted_date_phrase"),
-            pytest.param("title:202[0-1]*", id="bracket_class_wildcard"),
-        ],
-    )
     def test_fuzzy_mode_does_not_raise(
         self,
         query_index: tantivy.Index,
         settings,
-        raw_query: str,
     ) -> None:
-        # These are all valid whoosh grammar that tantivy's own query parser
-        # (used only by the fuzzy blend clause) cannot parse; the fuzzy
-        # clause must degrade gracefully instead of raising and failing the
-        # whole query. See _try_parse_fuzzy_query.
         settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.5
-        assert isinstance(parse_user_query(query_index, raw_query, UTC), tantivy.Query)
+        assert isinstance(parse_user_query(query_index, "invoice", UTC), tantivy.Query)
 
-    def test_date_keyword_resolves_without_raising(
+    def test_date_rewriting_applied_before_tantivy_parse(
         self,
         query_index: tantivy.Index,
     ) -> None:
-        # whoosh-compat's DateParserPlugin resolves "today" against the AST
-        # directly (no string rewrite to an ISO range happens anywhere in
-        # this pipeline); the emitted tantivy query must still build cleanly.
+        # created:today must be rewritten to an ISO range before Tantivy parses it;
+        # if passed raw, Tantivy would reject "today" as an invalid date value
         with time_machine.travel(datetime(2026, 3, 28, 12, 0, tzinfo=UTC), tick=False):
             q = parse_user_query(query_index, "created:today", UTC)
         assert isinstance(q, tantivy.Query)
@@ -114,58 +466,302 @@ class TestParseUserQuery:
     ) -> None:
         assert isinstance(parse_user_query(query_index, raw_query, UTC), tantivy.Query)
 
+    @pytest.mark.parametrize(
+        "raw_query",
+        [
+            # Partial date scalar (year only)
+            pytest.param("created:2020", id="created_year_scalar"),
+            # 8-digit compact date range in brackets
+            pytest.param(
+                "created:[20200101 TO 20201231]",
+                id="created_8digit_bracket_range",
+            ),
+            # Comma-separated field + date range (Whoosh v2 multi-clause syntax)
+            pytest.param(
+                "title:x,created:[2020 TO 2021]",
+                id="title_comma_created_range",
+            ),
+            # Field alias: type -> document_type
+            pytest.param("type:invoice", id="type_alias"),
+            # Multi-word date keyword
+            pytest.param("created:previous week", id="created_previous_week"),
+            # Full ISO datetime range
+            pytest.param(
+                "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z]",
+                id="created_iso_range",
+            ),
+            # Comma-separated ISO ranges (Whoosh v2 syntax)
+            pytest.param(
+                "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z],"
+                "added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]",
+                id="comma_iso_ranges",
+            ),
+        ],
+    )
+    def test_advanced_search_queries_do_not_raise(
+        self,
+        query_index: tantivy.Index,
+        raw_query: str,
+    ) -> None:
+        """
+        End-to-end: queries that the frontend sends must parse without raising.
+
+        This tests the full pipeline: translate_query -> tantivy parse_query.
+        Equivalent to asserting HTTP 200 (not 400) for each query form.
+        """
+        with time_machine.travel(datetime(2026, 6, 15, 12, 0, tzinfo=UTC), tick=False):
+            assert isinstance(
+                parse_user_query(query_index, raw_query, UTC),
+                tantivy.Query,
+            )
+
     def test_invalid_date_propagates_not_swallowed(
         self,
         query_index: tantivy.Index,
     ) -> None:
-        # parse_user_query never falls back to the raw query string on a parse
-        # error: a bad date diagnostic from whoosh-compat always maps to an
-        # InvalidDateQuery and must propagate, so the view can return a 400
-        # instead of silently parsing the raw (invalid) date.
+        # parse_user_query falls back to the raw query on unexpected translation
+        # errors, but an InvalidDateQuery is intentional and must propagate so the
+        # view can return a 400 instead of silently parsing the raw (invalid) date.
         with pytest.raises(InvalidDateQuery) as exc_info:
             parse_user_query(query_index, "created:202023", UTC)
         assert exc_info.value.field == "created"
         assert exc_info.value.value == "202023"
 
-    def test_invalid_number_raises_invalid_number_query(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        with pytest.raises(InvalidNumberQuery) as exc_info:
-            parse_user_query(query_index, "asn:notanumber", UTC)
-        assert exc_info.value.field == "asn"
-        assert exc_info.value.value == "notanumber"
 
-    def test_multiple_bad_fields_raise_multiple_search_query_errors(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        with pytest.raises(MultipleSearchQueryErrors) as exc_info:
-            parse_user_query(
-                query_index,
-                "created:notadate AND asn:notanumber",
-                UTC,
-            )
-        assert len(exc_info.value.errors) == 2
-        kinds = {type(e) for e in exc_info.value.errors}
-        assert kinds == {InvalidDateQuery, InvalidNumberQuery}
+class TestYearRangeRewriting:
+    """Whoosh-style year-only date ranges must be rewritten to ISO 8601."""
 
-    def test_unregistered_id_field_folds_to_literal_text_not_error(
+    @pytest.mark.parametrize(
+        ("query", "field", "expected_lo", "expected_hi"),
+        [
+            pytest.param(
+                "created:[2020 TO 2020]",
+                "created",
+                "2020-01-01T00:00:00Z",
+                "2021-01-01T00:00:00Z",
+                id="single_year_created",
+            ),
+            pytest.param(
+                "created:[2018 TO 2021]",
+                "created",
+                "2018-01-01T00:00:00Z",
+                "2022-01-01T00:00:00Z",
+                id="multi_year_range_created",
+            ),
+            pytest.param(
+                "added:[2022 TO 2023]",
+                "added",
+                "2022-01-01T00:00:00Z",
+                "2024-01-01T00:00:00Z",
+                id="added_field",
+            ),
+            pytest.param(
+                "modified:[2021 TO 2021]",
+                "modified",
+                "2021-01-01T00:00:00Z",
+                "2022-01-01T00:00:00Z",
+                id="modified_field",
+            ),
+            pytest.param(
+                "created:[2020 to 2020]",
+                "created",
+                "2020-01-01T00:00:00Z",
+                "2021-01-01T00:00:00Z",
+                id="lowercase_to_keyword",
+            ),
+        ],
+    )
+    def test_year_range_rewritten(
         self,
-        query_index: tantivy.Index,
+        query: str,
+        field: str,
+        expected_lo: str,
+        expected_hi: str,
     ) -> None:
-        # tag_id is intentionally excluded from the FieldRegistry: whoosh-compat
-        # parity leniency folds it into literal text, not a diagnostic/400.
-        # A result-level assertion that this fold actually matches nothing
-        # against real documents lives in
-        # test_acceptance.py::TestUnregisteredIdFieldFoldsToLiteralText.
-        q = parse_user_query(query_index, "tag_id:5", UTC)
-        assert isinstance(q, tantivy.Query)
+        result = translate_query(query, UTC)
+        lo, hi = _range(result, field)
+        assert lo == expected_lo
+        assert hi == expected_hi
+
+    def test_reversed_year_range_is_swapped(self) -> None:
+        # A reversed range must not yield lo > hi, which Tantivy treats as an
+        # empty range (silently zero results). The bounds are swapped instead.
+        result = translate_query("created:[2025 TO 2020]", UTC)
+        lo, hi = _range(result, "created")
+        assert lo == "2020-01-01T00:00:00Z"
+        assert hi == "2026-01-01T00:00:00Z"
+
+    def test_year_range_in_complex_boolean_query(self) -> None:
+        query = "tag:steuer AND (title:2020 OR (NOT title:2019 AND NOT title:2018 AND created:[2020 TO 2020]))"
+        result = translate_query(query, UTC)
+        lo, hi = _range(result, "created")
+        assert lo == "2020-01-01T00:00:00Z"
+        assert hi == "2021-01-01T00:00:00Z"
+        assert "title:2020" in result
+        assert "title:2019" in result
+        assert "title:2018" in result
+
+    def test_already_iso_date_range_passes_through_unchanged(self) -> None:
+        original = "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]"
+        assert translate_query(original, UTC) == original
+
+    def test_8digit_in_brackets_not_matched_as_year_range(self) -> None:
+        # [YYYYMMDD TO YYYYMMDD]: the translation layer converts 8-digit bounds to
+        # ISO day ranges. 20200101 -> 2020-01-01T00:00:00Z (lo of that day);
+        # 20201231 -> the ceil of Dec 31 = 2021-01-01T00:00:00Z (exclusive end).
+        # This is the correct and accepted behavior: old compact form becomes a
+        # proper Tantivy-parseable ISO range.
+        original = "created:[20200101 TO 20201231]"
+        result = translate_query(original, UTC)
+        lo, hi = _range(result, "created")
+        assert lo == "2020-01-01T00:00:00Z"
+        assert hi == "2021-01-01T00:00:00Z"
+
+
+class TestNonDateFieldsNotRewritten:
+    """Date rewriters must only fire on the date fields (created/modified/added).
+
+    Integer fields like asn/id/page_count and unknown fields would otherwise be
+    rewritten into date ranges and rejected by Tantivy as type mismatches.
+    """
+
+    @pytest.mark.parametrize(
+        "query",
+        [
+            pytest.param("asn:20240101", id="asn_8digit"),
+            pytest.param("id:20240101", id="id_8digit"),
+            pytest.param("page_count:12345678", id="page_count_8digit"),
+            pytest.param("num_notes:20231201", id="num_notes_8digit"),
+        ],
+    )
+    def test_8digit_on_integer_field_passes_through_unchanged(self, query: str) -> None:
+        assert translate_query(query, EASTERN) == query
+
+    @pytest.mark.parametrize(
+        "query",
+        [
+            pytest.param("asn:[2000 TO 2024]", id="asn_year_range"),
+            pytest.param("id:[2000 TO 2024]", id="id_year_range"),
+            pytest.param("page_count:[2000 TO 2024]", id="page_count_year_range"),
+        ],
+    )
+    def test_year_range_on_integer_field_passes_through_unchanged(
+        self,
+        query: str,
+    ) -> None:
+        assert translate_query(query, UTC) == query
+
+    def test_unknown_field_keyword_passes_through_unchanged(self) -> None:
+        # foobar is not a date field: 'foobar:today' must not become a date range,
+        # which Tantivy would otherwise reject as an unknown/typed field.
+        assert translate_query("foobar:today", UTC) == "foobar:today"
+
+
+class TestPassthrough:
+    """Queries without field prefixes or unrelated content pass through unchanged."""
+
+    def test_bare_keyword_no_field_prefix_unchanged(self) -> None:
+        # Bare 'today' with no field: prefix passes through unchanged
+        result = translate_query("bank statement today", UTC)
+        assert "today" in result
+
+    def test_unrelated_query_unchanged(self) -> None:
+        assert translate_query("title:invoice", UTC) == "title:invoice"
+
+
+class TestNormalizeQuery:
+    """translate_query expands comma-separated values and collapses whitespace."""
+
+    def test_normalize_expands_comma_separated_tags(self) -> None:
+        assert translate_query("tag:foo,bar", UTC) == "tag:foo AND tag:bar"
+
+    def test_normalize_comma_between_range_expressions(self) -> None:
+        # Comma-separated field range expressions (Whoosh v2 syntax) must be
+        # converted to AND so Tantivy does not receive an invalid comma.
+        q = "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z],added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        assert translate_query(q, UTC) == (
+            "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+            " AND "
+            "added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        )
+
+    def test_normalize_expands_three_values(self) -> None:
+        assert (
+            translate_query("tag:foo,bar,baz", UTC) == "tag:foo AND tag:bar AND tag:baz"
+        )
+
+    def test_normalize_collapses_whitespace(self) -> None:
+        assert translate_query("bank   statement", UTC) == "bank statement"
+
+    def test_normalize_no_commas_unchanged(self) -> None:
+        assert translate_query("bank statement", UTC) == "bank statement"
+
+    @pytest.mark.parametrize(
+        ("raw", "expected"),
+        [
+            pytest.param(
+                "h52.1 - kurzsichtigkeit",
+                "h52.1 kurzsichtigkeit",
+                id="icd_code_dash_description",
+            ),
+            pytest.param(
+                "H52.1 - asd",
+                "H52.1 asd",
+                id="icd_code_uppercase_dash",
+            ),
+            pytest.param(
+                "h52.1 -",
+                "h52.1",
+                id="trailing_minus",
+            ),
+            pytest.param(
+                ". -",
+                ".",
+                id="dot_trailing_minus",
+            ),
+            pytest.param(
+                "h52. -",
+                "h52.",
+                id="partial_code_trailing_minus",
+            ),
+            pytest.param(
+                "foo - bar - baz",
+                "foo bar baz",
+                id="multiple_dashes",
+            ),
+            pytest.param(
+                "foo + bar",
+                "foo bar",
+                id="spaced_plus_operator",
+            ),
+        ],
+    )
+    def test_normalize_strips_dangling_operators(self, raw: str, expected: str) -> None:
+        assert translate_query(raw, UTC) == expected
+
+    @pytest.mark.parametrize(
+        "query",
+        [
+            pytest.param("term -other", id="adjacent_not_operator"),
+            pytest.param("-term", id="leading_not_operator"),
+            pytest.param("+term", id="leading_must_operator"),
+            pytest.param("foo -bar +baz", id="mixed_adjacent_operators"),
+        ],
+    )
+    def test_normalize_preserves_valid_operators(self, query: str) -> None:
+        assert translate_query(query, UTC) == query
 
 
 class TestParseSimpleTextHighlightQuery:
     """parse_simple_text_highlight_query must not raise on natural-language queries."""
 
+    @pytest.fixture
+    def query_index(self) -> tantivy.Index:
+        schema = build_schema()
+        idx = tantivy.Index(schema, path=None)
+        register_tokenizers(idx, "")
+        return idx
+
     @pytest.mark.parametrize(
         "raw_query",
         [
@@ -187,25 +783,16 @@ class TestParseSimpleTextHighlightQuery:
             tantivy.Query,
         )
 
-    def test_a_real_token_matches_the_corpus(
-        self,
-        populated_index: tantivy.Index,
-    ) -> None:
-        """Without this, an empty corpus would make the two assertions below
-        pass for a query that matches every document."""
-        assert _highlight_hit_count(populated_index, "needle") == 1
+    def test_empty_query_returns_empty_query(self, query_index: tantivy.Index) -> None:
+        result = parse_simple_text_highlight_query(query_index, "")
+        assert isinstance(result, tantivy.Query)
 
-    def test_empty_query_matches_no_document(
+    def test_all_operators_returns_empty_query(
         self,
-        populated_index: tantivy.Index,
+        query_index: tantivy.Index,
     ) -> None:
-        assert _highlight_hit_count(populated_index, "") == 0
-
-    def test_all_operators_query_matches_no_document(
-        self,
-        populated_index: tantivy.Index,
-    ) -> None:
-        assert _highlight_hit_count(populated_index, "- +") == 0
+        result = parse_simple_text_highlight_query(query_index, "- +")
+        assert isinstance(result, tantivy.Query)
 
 
 class TestPermissionFilter:
@@ -297,52 +884,3 @@ class TestPermissionFilter:
         user = django_user_model(pk=20)
         perm = build_permission_filter(perm_index.schema, user)
         assert perm_index.searcher().search(perm, limit=10).count == 1  # only unowned
-
-
-class TestSearchQueryErrors:
-    def test_invalid_date_query_is_a_search_query_error(self) -> None:
-        err = InvalidDateQuery("created", "notadate")
-        assert isinstance(err, SearchQueryError)
-        assert err.field == "created"
-        assert err.value == "notadate"
-        assert "created" in str(err)
-        assert "notadate" in str(err)
-
-    def test_invalid_number_query_is_a_search_query_error(self) -> None:
-        err = InvalidNumberQuery("asn", "notanumber")
-        assert isinstance(err, SearchQueryError)
-        assert err.field == "asn"
-        assert err.value == "notanumber"
-        assert "asn" in str(err)
-        assert "notanumber" in str(err)
-
-    def test_multiple_search_query_errors_aggregates(self) -> None:
-        sub_errors = [
-            InvalidDateQuery("created", "notadate"),
-            InvalidNumberQuery("asn", "notanumber"),
-        ]
-        err = MultipleSearchQueryErrors(sub_errors)
-        assert isinstance(err, SearchQueryError)
-        assert err.errors == tuple(sub_errors)
-        assert "created" in str(err)
-        assert "asn" in str(err)
-
-
-class TestEmitErrorContract:
-    """A QueryError from emit() surfaces as a SearchQueryError (HTTP 400).
-
-    The Cause-based routing table itself is covered in test_error_routing.py.
-    """
-
-    def test_exists_requires_fast_gets_the_user_facing_rewrite(
-        self,
-        query_index: tantivy.Index,
-    ) -> None:
-        # whoosh-compat's own message advises a host-side fast=True config
-        # change the user can't act on, so this checks OUR wording, not
-        # whoosh-compat's (that's its own test suite's job now).
-        with pytest.raises(SearchQueryError) as exc_info:
-            parse_user_query(query_index, "notes.user:*", UTC)
-        assert str(exc_info.value) == (
-            "Existence searches (field:*) are not supported for field 'notes.user'."
-        )

src/documents/tests/search/test_query_negation.py

@@ -1,153 +0,0 @@
-"""Negation must survive the blended query.
-
-parse_user_query ORs an exact clause with optional fuzzy and CJK clauses.
-Each of those is built from positive terms only, so unless the query's
-exclusions are applied to the blend as a whole, a document the exact
-clause excluded is re-admitted by whichever other clause is enabled.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-
-from documents.models import Document
-
-if TYPE_CHECKING:
-    from pytest_django.fixtures import SettingsWrapper
-
-    from documents.search._backend import TantivyBackend
-
-pytestmark = [pytest.mark.search, pytest.mark.django_db]
-
-
-def _matched_ids(backend: TantivyBackend, query: str) -> set[int]:
-    return set(backend.search_ids(query, user=None))
-
-
-def _index(backend: TantivyBackend, **kwargs: object) -> Document:
-    doc = Document.objects.create(**kwargs)
-    backend.add_or_update(doc)
-    return doc
-
-
-@pytest.fixture
-def fuzzy_enabled(settings: SettingsWrapper) -> None:
-    """Enable the fuzzy blend clause. The threshold doubles as a minimum
-    score filter, so it is set to 0.0: every hit passes and the test sees
-    the clause's matching behaviour, not the filter's."""
-    settings.ADVANCED_FUZZY_SEARCH_THRESHOLD = 0.0
-
-
-class TestNegationConstrainsEveryClause:
-    @pytest.mark.usefixtures("fuzzy_enabled")
-    def test_fuzzy_clause_does_not_readmit_an_excluded_document(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        secret = _index(
-            backend,
-            title="Invoice A",
-            content="invoice total secret",
-            checksum="neg-fuzzy-1",
-        )
-        public = _index(
-            backend,
-            title="Invoice B",
-            content="invoice total public",
-            checksum="neg-fuzzy-2",
-        )
-
-        assert _matched_ids(backend, "invoice") == {secret.pk, public.pk}
-        assert _matched_ids(backend, "invoice NOT secret") == {public.pk}
-
-    def test_cjk_clause_does_not_readmit_an_excluded_document(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """The CJK clause legitimately carries 東京 here, so rebuilding it
-        from the AST cannot help: only applying the exclusion above the
-        blend keeps the secret document out."""
-        secret = _index(
-            backend,
-            title="Tokyo A",
-            content="東京都の秘密です secret",
-            checksum="neg-cjk-1",
-        )
-        public = _index(
-            backend,
-            title="Tokyo B",
-            content="東京都の報告書です public",
-            checksum="neg-cjk-2",
-        )
-
-        assert _matched_ids(backend, "東京") == {secret.pk, public.pk}
-        assert _matched_ids(backend, "東京 NOT secret") == {public.pk}
-
-    @pytest.mark.usefixtures("fuzzy_enabled")
-    def test_disjunctive_negation_still_admits_the_other_branch(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """'invoice OR NOT secret' excludes nothing on its own: a document
-        matching the left branch stays in even though it contains secret."""
-        secret_invoice = _index(
-            backend,
-            title="Invoice A",
-            content="invoice total secret",
-            checksum="neg-or-1",
-        )
-        unrelated = _index(
-            backend,
-            title="Recipe",
-            content="flour and water",
-            checksum="neg-or-2",
-        )
-
-        assert _matched_ids(backend, "invoice OR NOT secret") == {
-            secret_invoice.pk,
-            unrelated.pk,
-        }
-
-    def test_a_negation_under_or_does_not_constrain_the_cjk_clause(
-        self,
-        backend: TantivyBackend,
-    ) -> None:
-        """The limit of the hoist, pinned deliberately.
-
-        An exclusion that is one branch's own condition cannot be restated
-        above the blend without dropping documents the other branch
-        matches, so it is left where it is and the CJK clause stays
-        unconstrained by it. That shows through here in a way it does not
-        for latin text: the exact clause cannot match a CJK run at all, so
-        the CJK clause is the only thing matching the tokyo documents, and
-        the secret one comes with it.
-        """
-        secret = _index(
-            backend,
-            title="Tokyo A",
-            content="東京都の秘密です secret",
-            checksum="neg-or-cjk-1",
-        )
-        public = _index(
-            backend,
-            title="Tokyo B",
-            content="東京都の報告書です public",
-            checksum="neg-or-cjk-2",
-        )
-        bill = _index(
-            backend,
-            title="Bill",
-            content="bill payment received",
-            checksum="neg-or-cjk-3",
-        )
-
-        assert _matched_ids(backend, "(東京 AND NOT secret) OR bill") == {
-            bill.pk,
-            public.pk,
-            secret.pk,
-        }
-        # The same exclusion in conjunctive position is hoisted, and does
-        # constrain the CJK clause.
-        assert _matched_ids(backend, "東京 AND NOT secret") == {public.pk}

src/documents/tests/search/test_registry.py

@@ -1,149 +0,0 @@
-from collections.abc import Sequence
-
-import pytest
-from whoosh_compat import FieldKind
-from whoosh_compat import FieldRegistry
-from whoosh_compat.fields import ResolvedField
-
-from documents.search._fields import PUBLIC_FIELDS
-from documents.search._registry import get_field_registry
-
-
-@pytest.fixture
-def registry() -> FieldRegistry:
-    return get_field_registry(None)
-
-
-def _resolve(registry: FieldRegistry, name: str) -> ResolvedField:
-    ref = registry.make_ref(name)
-    assert ref is not None, f"{name} is not a valid field ref"
-    resolved = registry.resolve(ref)
-    assert resolved is not None, f"{name} did not resolve"
-    return resolved
-
-
-def _distinct_forms(result: str | Sequence[str]) -> tuple[str, ...]:
-    """The forms a term may match, in order, the way whoosh-compat's emitter
-    reads a pattern_normalizer's answer: a bare str is one form, a sequence is
-    several, deduplicated."""
-    if isinstance(result, str):
-        return (result,)
-    return tuple(dict.fromkeys(result))
-
-
-class TestFieldRegistry:
-    def test_internal_id_fields_are_not_registered(
-        self,
-        registry: FieldRegistry,
-    ) -> None:
-        for name in (
-            "tag_id",
-            "owner_id",
-            "viewer_id",
-            "correspondent_id",
-            "document_type_id",
-            "storage_path_id",
-            "viewer_group_id",
-        ):
-            assert name not in registry
-
-    def test_no_queryable_field_name_ends_in_id(self) -> None:
-        # The list above names the seven that were dropped; this catches the
-        # eighth. Internal *_id columns are written for permission filtering
-        # and joins, and whoosh only exposed them as query fields by accident,
-        # so a new one reaching the query surface is a leak rather than a
-        # feature. Checked against PUBLIC_FIELDS rather than the registry so
-        # an internal field is caught where it is declared.
-        leaked = [f.name for f in PUBLIC_FIELDS if f.name.endswith("_id")]
-        assert not leaked, f"internal id fields reached the query surface: {leaked}"
-
-    def test_type_alias_resolves_to_document_type(
-        self,
-        registry: FieldRegistry,
-    ) -> None:
-        assert _resolve(registry, "type").spec.name == "document_type"
-
-    def test_path_alias_resolves_to_storage_path(self, registry: FieldRegistry) -> None:
-        assert _resolve(registry, "path").spec.name == "storage_path"
-
-    def test_notes_json_subpaths_resolve(self, registry: FieldRegistry) -> None:
-        resolved = _resolve(registry, "notes.user")
-        assert resolved.spec.name == "notes"
-        assert resolved.json_path == "user"
-        assert resolved.is_subpath is True
-
-    def test_custom_fields_json_subpaths_resolve(self, registry: FieldRegistry) -> None:
-        for raw in ("custom_fields.name", "custom_fields.value"):
-            _resolve(registry, raw)
-
-    def test_unregistered_json_subpath_does_not_resolve(
-        self,
-        registry: FieldRegistry,
-    ) -> None:
-        # An unregistered subpath is not even a valid FieldRef: make_ref
-        # returns None for a dotted name whose subpath isn't registered
-        # (it doesn't produce a ref for resolve() to then reject).
-        assert registry.make_ref("notes.bogus") is None
-
-    def test_tag_is_comma_values(self, registry: FieldRegistry) -> None:
-        assert _resolve(registry, "tag").spec.comma_values is True
-
-    def test_correspondent_is_not_comma_values(self, registry: FieldRegistry) -> None:
-        # "tag" is the only field that opts in. This is only observable here:
-        # end to end the two readings of "correspondent:foo,bar" agree,
-        # because the analyzer splits the literal value on the comma anyway,
-        # so a result-level test cannot tell a value list from literal text.
-        assert _resolve(registry, "correspondent").spec.comma_values is False
-
-    def test_created_is_date_kind(self, registry: FieldRegistry) -> None:
-        resolved = _resolve(registry, "created")
-        assert resolved.spec.kind is FieldKind.DATE
-        assert resolved.spec.date_only is True
-
-    def test_analyzer_lowercases_and_ascii_folds(self, registry: FieldRegistry) -> None:
-        # title uses the paperless_text analyzer: simple -> remove_long ->
-        # lowercase -> ascii_fold [-> stemmer]. With no language configured
-        # (None), no stemmer runs, so "Café" folds to the single token "cafe".
-        resolved = _resolve(registry, "title")
-        assert resolved.spec.analyzer is not None
-        assert resolved.spec.analyzer("Café") == ["cafe"]
-
-    def test_checksum_analyzer_is_identity_single_token(
-        self,
-        registry: FieldRegistry,
-    ) -> None:
-        # checksum uses the raw tokenizer at index time (no splitting).
-        resolved = _resolve(registry, "checksum")
-        assert resolved.spec.analyzer is not None
-        assert resolved.spec.analyzer("ABC-123") == ["ABC-123"]
-
-    def test_pattern_normalizer_follows_the_registry_language(
-        self,
-        registry: FieldRegistry,
-    ) -> None:
-        # Index terms are stemmed, so patterns offer their stem too, using the
-        # registry's own language: "Running" has to reach the indexed "run".
-        # Without a language the index holds surface forms, so there is no
-        # second form and the run is only case/accent-folded.
-        resolved = _resolve(registry, "title")
-        assert resolved.spec.pattern_normalizer is not None
-        assert _distinct_forms(resolved.spec.pattern_normalizer("Running")) == (
-            "running",
-        )
-
-        resolved_en = _resolve(get_field_registry("en"), "title")
-        assert resolved_en.spec.pattern_normalizer is not None
-        assert _distinct_forms(resolved_en.spec.pattern_normalizer("Running")) == (
-            "running",
-            "run",
-        )
-
-    def test_registry_is_cached_per_language(self) -> None:
-        a = get_field_registry("en")
-        b = get_field_registry("en")
-        assert a is b
-
-    def test_registry_rebuilds_on_language_change(self) -> None:
-        a = get_field_registry("en")
-        b = get_field_registry("de")
-        assert a is not b

src/documents/tests/search/test_schema.py

@@ -1,20 +1,12 @@
 from __future__ import annotations
 
 import json
-from datetime import UTC
-from datetime import datetime
 from typing import TYPE_CHECKING
 
 import pytest
-import tantivy
 
-from documents.search._fields import PUBLIC_FIELDS
 from documents.search._schema import SCHEMA_VERSION
-from documents.search._schema import build_schema
-from documents.search._schema import field_descriptors
 from documents.search._schema import needs_rebuild
-from documents.search._schema import schema_fingerprint
-from documents.search._tokenizer import register_tokenizers
 
 if TYPE_CHECKING:
     from pathlib import Path
@@ -37,13 +29,7 @@ class TestNeedsRebuild:
     ) -> None:
         settings.SEARCH_LANGUAGE = "en"
         (index_dir / ".index_settings.json").write_text(
-            json.dumps(
-                {
-                    "schema_version": SCHEMA_VERSION,
-                    "language": "en",
-                    "schema_fingerprint": schema_fingerprint(),
-                },
-            ),
+            json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}),
         )
         assert needs_rebuild(index_dir) is False
 
@@ -90,72 +76,3 @@ class TestNeedsRebuild:
             json.dumps({"schema_version": SCHEMA_VERSION, "language": "en"}),
         )
         assert needs_rebuild(index_dir) is True
-
-
-def _schema_fields(schema: tantivy.Schema) -> dict[str, dict]:
-    """{name: field-state} for every field declared on a tantivy Schema.
-
-    tantivy-py 0.26 exposes no public introspection API on Schema (no
-    __iter__, get_field, to_json, etc.) -- __reduce__() (used internally for
-    pickling) is the only way to recover the field list, so we lean on it
-    here for test assertions only.
-    """
-    state = schema.__reduce__()[1][0]
-    return {field["name"]: field for field in state["inner"]}
-
-
-class TestSchemaMatchesPublicFields:
-    def test_every_public_field_is_in_the_schema(self) -> None:
-        schema = build_schema()
-        schema_field_names = set(_schema_fields(schema))
-        for field in PUBLIC_FIELDS:
-            assert field.name in schema_field_names, (
-                f"{field.name} is in PUBLIC_FIELDS but missing from build_schema()"
-            )
-
-    def test_asn_page_count_num_notes_are_fast_unsigned_fields(self) -> None:
-        # Spot-check kind-derived construction for the U64 fields.
-        schema = build_schema()
-        doc = tantivy.Document()
-        doc.add_unsigned("id", 1)
-        doc.add_text("checksum", "x")
-        doc.add_unsigned("asn", 42)
-        doc.add_unsigned("page_count", 3)
-        doc.add_unsigned("num_notes", 0)
-        doc.add_date("created", datetime(2020, 1, 1, tzinfo=UTC))
-        doc.add_date("modified", datetime(2020, 1, 1, tzinfo=UTC))
-        doc.add_date("added", datetime(2020, 1, 1, tzinfo=UTC))
-        index = tantivy.Index(schema)
-        register_tokenizers(index, None)
-        writer = index.writer()
-        writer.add_document(doc)
-        writer.commit()
-        index.reload()
-        searcher = index.searcher()
-        results = searcher.search(tantivy.Query.term_query(schema, "asn", 42), limit=1)
-        assert len(results.hits) == 1
-
-
-class TestFastFlagAgreement:
-    def test_every_public_field_fast_flag_matches_the_built_schema(self) -> None:
-        # whoosh-compat's registry trusts PUBLIC_FIELDS' fast flag when resolving
-        # field:* existence checks (its FAST_FIELD strategy); a fast=True
-        # entry whose actual tantivy column is not fast would make those
-        # searches silently match nothing at search time. Only the U64 and
-        # DATE descriptors can carry the flag today, so this
-        # pins the agreement for EVERY kind: a future fast=True
-        # TEXT/KEYWORD/JSON entry the builder silently ignores fails here
-        # instead of at a user's query.
-        #
-        # field_descriptors() (not tantivy-py's __reduce__() pickling
-        # internals) is used as the probe here: it is exactly the input
-        # build_schema()'s SchemaBuilder consumes for the `fast` kwarg on
-        # every field kind, so it pins the same agreement without depending
-        # on a private pickled representation surviving a tantivy-py
-        # upgrade.
-        descriptor_fast = {d.name: d.fast for d in field_descriptors()}
-        for public_field in PUBLIC_FIELDS:
-            assert descriptor_fast[public_field.name] == public_field.fast, (
-                f"{public_field.name}: PUBLIC_FIELDS says fast={public_field.fast} but"
-                f" field_descriptors() says fast={descriptor_fast[public_field.name]}"
-            )

src/documents/tests/search/test_schema_fingerprint.py

@@ -1,492 +0,0 @@
-"""The schema fingerprint stamped into .index_settings.json.
-
-tantivy compares schemas by *ordered* field list, and `tantivy.Index(schema,
-path=...)` (what every write path does) raises on any difference. SCHEMA_VERSION
-is the manual guard against that, but build_schema() is edited for *parser*
-reasons - adding an alias, flipping fast=True, adding a subpath - by people not
-thinking about the on-disk index, and forgetting the bump is exactly how this
-branch's bug happened.
-
-The fingerprint is the automatic guard: it hashes the field descriptor list that
-build_schema() itself iterates, so any change to a field's name, kind, options
-or *position* forces a rebuild on its own.
-"""
-
-from __future__ import annotations
-
-import hashlib
-import json
-from typing import TYPE_CHECKING
-
-import pytest
-import tantivy
-
-from documents.search import _schema
-from documents.search._schema import SCHEMA_VERSION
-from documents.search._schema import FieldDescriptor
-from documents.search._schema import _write_sentinels
-from documents.search._schema import build_schema
-from documents.search._schema import field_descriptors
-from documents.search._schema import needs_rebuild
-from documents.search._schema import schema_fingerprint
-
-if TYPE_CHECKING:
-    from pathlib import Path
-
-    from pytest_django.fixtures import SettingsWrapper
-
-pytestmark = pytest.mark.search
-
-# The on-disk field layout of a v2 index, pinned as data. Any edit here is an
-# index-format change: it must come with a rebuild, which the fingerprint now
-# forces automatically. Reproduced from build_schema()'s output as it stood
-# before the descriptor refactor, so it also pins that the refactor changed
-# nothing.
-PINNED_DESCRIPTORS: tuple[FieldDescriptor, ...] = (
-    FieldDescriptor("id", "u64", stored=True, indexed=True, fast=True, tokenizer=None),
-    FieldDescriptor(
-        "title",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "content",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "correspondent",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "document_type",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "storage_path",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "original_filename",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "tag",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "checksum",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="raw",
-    ),
-    FieldDescriptor("asn", "u64", stored=True, indexed=True, fast=True, tokenizer=None),
-    FieldDescriptor(
-        "page_count",
-        "u64",
-        stored=True,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "num_notes",
-        "u64",
-        stored=True,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "created",
-        "date",
-        stored=True,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "modified",
-        "date",
-        stored=True,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "added",
-        "date",
-        stored=True,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "notes",
-        "json",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "notes_text",
-        "text",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "custom_fields",
-        "json",
-        stored=True,
-        indexed=True,
-        fast=False,
-        tokenizer="paperless_text",
-    ),
-    FieldDescriptor(
-        "title_sort",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer="simple_analyzer",
-    ),
-    FieldDescriptor(
-        "correspondent_sort",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer="simple_analyzer",
-    ),
-    FieldDescriptor(
-        "type_sort",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer="simple_analyzer",
-    ),
-    FieldDescriptor(
-        "bigram_content",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="bigram_analyzer",
-    ),
-    FieldDescriptor(
-        "bigram_title",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="bigram_analyzer",
-    ),
-    FieldDescriptor(
-        "bigram_correspondent",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="bigram_analyzer",
-    ),
-    FieldDescriptor(
-        "bigram_document_type",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="bigram_analyzer",
-    ),
-    FieldDescriptor(
-        "bigram_tag",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="bigram_analyzer",
-    ),
-    FieldDescriptor(
-        "simple_title",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="simple_search_analyzer",
-    ),
-    FieldDescriptor(
-        "simple_content",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="simple_search_analyzer",
-    ),
-    FieldDescriptor(
-        "autocomplete_word",
-        "text",
-        stored=False,
-        indexed=True,
-        fast=False,
-        tokenizer="raw",
-    ),
-    FieldDescriptor(
-        "owner_id",
-        "u64",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "viewer_id",
-        "u64",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-    FieldDescriptor(
-        "viewer_group_id",
-        "u64",
-        stored=False,
-        indexed=True,
-        fast=True,
-        tokenizer=None,
-    ),
-)
-
-
-def _schema_fields(schema: tantivy.Schema) -> list[dict]:
-    """The tantivy-level field list, in declaration order.
-
-    tantivy-py 0.26 exposes no public introspection API on Schema, so
-    __reduce__() (its pickling hook) is the only way to recover the field list.
-    It is used here, in a test, precisely because it is the representation the
-    persisted fingerprint must NOT depend on.
-    """
-    return schema.__reduce__()[1][0]["inner"]
-
-
-def _sentinels(index_dir: Path, **overrides: object) -> None:
-    data = {
-        "schema_version": SCHEMA_VERSION,
-        "language": None,
-        "schema_fingerprint": schema_fingerprint(),
-    }
-    data.update(overrides)
-    (index_dir / ".index_settings.json").write_text(json.dumps(data))
-
-
-class TestDescriptorsDescribeTheBuiltSchema:
-    def test_descriptors_match_the_pinned_field_layout(self) -> None:
-        assert tuple(field_descriptors()) == PINNED_DESCRIPTORS
-
-    def test_built_schema_matches_the_descriptors(self) -> None:
-        """The descriptors are not a parallel description - they are the input.
-
-        Reading the built schema back proves the loop honours every option, so
-        a descriptor edit cannot claim a shape the SchemaBuilder did not build.
-        """
-        kinds = {"text": "text", "json": "json_object", "u64": "u64", "date": "date"}
-        built = [
-            (
-                field["name"],
-                field["type"],
-                field["options"]["stored"],
-                bool(field["options"].get("fast")),
-                (field["options"].get("indexing") or {}).get("tokenizer"),
-            )
-            for field in _schema_fields(build_schema())
-        ]
-        expected = [
-            (
-                descriptor.name,
-                kinds[descriptor.kind],
-                descriptor.stored,
-                descriptor.fast,
-                descriptor.tokenizer,
-            )
-            for descriptor in field_descriptors()
-        ]
-        assert built == expected
-
-
-class TestFingerprintSensitivity:
-    def test_a_field_option_change_moves_the_fingerprint(
-        self,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        before = schema_fingerprint()
-        changed = field_descriptors()
-        changed[1] = changed[1]._replace(fast=True)
-        monkeypatch.setattr(_schema, "field_descriptors", lambda: changed)
-
-        assert schema_fingerprint() != before
-
-    def test_reordering_alone_moves_the_fingerprint(
-        self,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        """The original bug: same fields, different declaration order.
-
-        A set- or dict-based fingerprint would be blind to this, and tantivy
-        would reject every write against the existing index.
-        """
-        before = schema_fingerprint()
-        swapped = field_descriptors()
-        swapped[1], swapped[2] = swapped[2], swapped[1]
-        monkeypatch.setattr(_schema, "field_descriptors", lambda: swapped)
-
-        assert schema_fingerprint() != before
-
-    def test_repeated_calls_agree(self) -> None:
-        assert schema_fingerprint() == schema_fingerprint()
-
-
-class TestFingerprintIsIndependentOfTantivy:
-    def test_a_tantivy_option_key_addition_would_not_move_it(self) -> None:
-        """A tantivy-py upgrade must not force a global reindex.
-
-        Hashing schema.__reduce__() would do exactly that: the simulated new
-        option key below changes that payload for every user with no schema
-        change at all.
-        """
-        fields = _schema_fields(build_schema())
-        upgraded = [
-            {**field, "options": {**field["options"], "coerce": True}}
-            for field in fields
-        ]
-        assert _hash(upgraded) != _hash(fields)
-        assert schema_fingerprint() == _fingerprint_of(field_descriptors())
-
-    def test_fingerprint_never_touches_the_schema_builder(
-        self,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        before = schema_fingerprint()
-
-        class _RemovedSchemaBuilder:
-            def __init__(self) -> None:
-                raise AssertionError("tantivy.SchemaBuilder was consulted")
-
-        monkeypatch.setattr(tantivy, "SchemaBuilder", _RemovedSchemaBuilder)
-        with pytest.raises(AssertionError):
-            build_schema()
-
-        assert schema_fingerprint() == before
-
-
-def _hash(payload: object) -> str:
-    return hashlib.blake2b(json.dumps(payload).encode()).hexdigest()
-
-
-def _fingerprint_of(descriptors: list[FieldDescriptor]) -> str:
-    return _hash([list(descriptor) for descriptor in descriptors])
-
-
-class TestNeedsRebuildOnFingerprint:
-    def test_matching_fingerprint_does_not_rebuild(
-        self,
-        index_dir: Path,
-        settings: SettingsWrapper,
-    ) -> None:
-        settings.SEARCH_LANGUAGE = None
-        _sentinels(index_dir)
-
-        assert needs_rebuild(index_dir) is False
-
-    def test_stale_fingerprint_rebuilds_despite_a_matching_version(
-        self,
-        index_dir: Path,
-        settings: SettingsWrapper,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        """The failure this task exists to prevent: schema edited, version not
-        bumped. Without the fingerprint check, `reindex --if-needed` reports the
-        index up to date and every write then raises."""
-        settings.SEARCH_LANGUAGE = None
-        _sentinels(index_dir)
-        extended = [
-            *field_descriptors(),
-            FieldDescriptor(
-                "new_field",
-                "u64",
-                stored=False,
-                indexed=True,
-                fast=True,
-                tokenizer=None,
-            ),
-        ]
-        monkeypatch.setattr(_schema, "field_descriptors", lambda: extended)
-
-        assert needs_rebuild(index_dir) is True
-
-    def test_reordered_schema_rebuilds(
-        self,
-        index_dir: Path,
-        settings: SettingsWrapper,
-        monkeypatch: pytest.MonkeyPatch,
-    ) -> None:
-        settings.SEARCH_LANGUAGE = None
-        _sentinels(index_dir)
-        reordered = field_descriptors()
-        reordered[1], reordered[2] = reordered[2], reordered[1]
-        monkeypatch.setattr(_schema, "field_descriptors", lambda: reordered)
-
-        assert needs_rebuild(index_dir) is True
-
-    def test_missing_fingerprint_rebuilds(
-        self,
-        index_dir: Path,
-        settings: SettingsWrapper,
-    ) -> None:
-        """No seeding: an index whose schema shape nobody recorded is rebuilt
-        rather than trusted."""
-        settings.SEARCH_LANGUAGE = None
-        (index_dir / ".index_settings.json").write_text(
-            json.dumps({"schema_version": SCHEMA_VERSION, "language": None}),
-        )
-
-        assert needs_rebuild(index_dir) is True
-
-    def test_written_sentinels_satisfy_the_check(
-        self,
-        index_dir: Path,
-        settings: SettingsWrapper,
-    ) -> None:
-        settings.SEARCH_LANGUAGE = "en"
-        _write_sentinels(index_dir)
-
-        assert needs_rebuild(index_dir) is False

src/documents/tests/search/test_schema_version.py

@@ -1,164 +0,0 @@
-"""SCHEMA_VERSION must change whenever build_schema()'s field list or order does.
-
-tantivy compares schemas by *ordered* field list. ``Index.open()`` loads the
-schema from the index's own ``meta.json``, so reads against an index built by an
-older release keep working after a field reorder. Writes do not:
-``WriteBatch.__enter__`` calls ``tantivy.Index(build_schema(), path=...)``, an
-open-or-create that raises ``ValueError`` on any schema difference. Nothing
-catches that ValueError, so consumption, index_document and bulk edit all
-hard-fail while ``/api/status/`` still reports the index healthy.
-
-The only thing that saves such an install is ``needs_rebuild()`` noticing the
-version stamped in ``.index_settings.json`` is stale.
-"""
-
-from __future__ import annotations
-
-import json
-from typing import TYPE_CHECKING
-
-import pytest
-import tantivy
-from django.conf import settings as django_settings
-
-from documents.search._schema import build_schema
-from documents.search._schema import needs_rebuild
-from documents.search._schema import open_or_rebuild_index
-
-if TYPE_CHECKING:
-    from pathlib import Path
-
-pytestmark = [pytest.mark.search]
-
-RELEASED_V1_SCHEMA_VERSION = 1
-
-
-def _build_released_v1_schema() -> tantivy.Schema:
-    """Frozen copy of build_schema() as shipped in v3.0.x (schema version 1).
-
-    Deliberately duplicated rather than imported: it must keep describing the
-    on-disk layout of already-deployed indexes even as build_schema() evolves.
-    """
-    sb = tantivy.SchemaBuilder()
-
-    sb.add_unsigned_field("id", stored=True, indexed=True, fast=True)
-    sb.add_text_field("checksum", stored=True, tokenizer_name="raw")
-
-    for field in (
-        "title",
-        "correspondent",
-        "document_type",
-        "storage_path",
-        "original_filename",
-        "content",
-    ):
-        sb.add_text_field(field, stored=True, tokenizer_name="paperless_text")
-
-    for field in ("title_sort", "correspondent_sort", "type_sort"):
-        sb.add_text_field(
-            field,
-            stored=False,
-            tokenizer_name="simple_analyzer",
-            fast=True,
-        )
-
-    for field in (
-        "bigram_content",
-        "bigram_title",
-        "bigram_correspondent",
-        "bigram_document_type",
-        "bigram_tag",
-    ):
-        sb.add_text_field(field, stored=False, tokenizer_name="bigram_analyzer")
-
-    for field in ("simple_title", "simple_content"):
-        sb.add_text_field(field, stored=False, tokenizer_name="simple_search_analyzer")
-
-    sb.add_text_field("autocomplete_word", stored=False, tokenizer_name="raw")
-    sb.add_text_field("tag", stored=True, tokenizer_name="paperless_text")
-
-    sb.add_json_field("notes", stored=True, tokenizer_name="paperless_text")
-    sb.add_text_field("notes_text", stored=True, tokenizer_name="paperless_text")
-    sb.add_json_field("custom_fields", stored=True, tokenizer_name="paperless_text")
-
-    for field in (
-        "correspondent_id",
-        "document_type_id",
-        "storage_path_id",
-        "tag_id",
-        "owner_id",
-        "viewer_id",
-        "viewer_group_id",
-    ):
-        sb.add_unsigned_field(field, stored=False, indexed=True, fast=True)
-
-    for field in ("created", "modified", "added"):
-        sb.add_date_field(field, stored=True, indexed=True, fast=True)
-
-    for field in ("asn", "page_count", "num_notes"):
-        sb.add_unsigned_field(field, stored=True, indexed=True, fast=True)
-
-    return sb.build()
-
-
-@pytest.fixture
-def released_v1_index(tmp_path: Path) -> Path:
-    """An index directory as a v3.0.x install would leave it on disk."""
-    index_dir = tmp_path / "index"
-    index_dir.mkdir()
-    tantivy.Index(_build_released_v1_schema(), path=str(index_dir))
-    (index_dir / ".index_settings.json").write_text(
-        json.dumps(
-            {
-                "schema_version": RELEASED_V1_SCHEMA_VERSION,
-                "language": django_settings.SEARCH_LANGUAGE,
-            },
-        ),
-    )
-    return index_dir
-
-
-class TestUpgradeFromReleasedV1Index:
-    def test_released_v1_index_is_flagged_for_rebuild(
-        self,
-        released_v1_index: Path,
-    ) -> None:
-        """The current schema differs from v1's, so the sentinel must be stale.
-
-        If this fails, `document_index reindex --if-needed` prints "Search index
-        is up to date" and skips, leaving the mismatched index in place.
-        """
-        assert needs_rebuild(released_v1_index) is True
-
-    def test_v1_index_rejects_writes_against_the_current_schema(
-        self,
-        released_v1_index: Path,
-    ) -> None:
-        """The failure mode the version bump exists to prevent.
-
-        This is exactly what WriteBatch.__enter__ does on every index write.
-        """
-        with pytest.raises(ValueError, match="schema does not match"):
-            tantivy.Index(build_schema(), path=str(released_v1_index))
-
-    def test_opening_a_v1_index_leaves_it_writable(
-        self,
-        released_v1_index: Path,
-    ) -> None:
-        """End to end: open_or_rebuild_index must hand back an index that the
-        write path can reopen. Before the version bump, needs_rebuild() returned
-        False here, the stale directory survived untouched, and every subsequent
-        write raised the ValueError above."""
-        open_or_rebuild_index(released_v1_index)
-
-        tantivy.Index(build_schema(), path=str(released_v1_index))
-
-    def test_rebuilt_index_is_not_rebuilt_again(
-        self,
-        released_v1_index: Path,
-    ) -> None:
-        """The rebuild must stamp the version it actually wrote, otherwise every
-        startup wipes and reindexes the whole corpus."""
-        open_or_rebuild_index(released_v1_index)
-
-        assert needs_rebuild(released_v1_index) is False

src/documents/tests/search/test_tokenizer.py

@@ -7,8 +7,8 @@ import pytest
 import tantivy
 
 from documents.search._tokenizer import _bigram_analyzer
+from documents.search._tokenizer import _paperless_text
 from documents.search._tokenizer import _simple_search_analyzer
-from documents.search._tokenizer import paperless_text_analyzer
 from documents.search._tokenizer import register_tokenizers
 
 if TYPE_CHECKING:
@@ -25,7 +25,7 @@ class TestTokenizers:
         sb.add_text_field("content", stored=True, tokenizer_name="paperless_text")
         schema = sb.build()
         idx = tantivy.Index(schema, path=None)
-        idx.register_tokenizer("paperless_text", paperless_text_analyzer(""))
+        idx.register_tokenizer("paperless_text", _paperless_text(""))
         return idx
 
     @pytest.fixture

src/documents/tests/search/test_translate.py

@@ -0,0 +1,810 @@
+from __future__ import annotations
+
+from datetime import UTC
+from datetime import datetime
+from typing import TYPE_CHECKING
+from zoneinfo import ZoneInfo
+
+import pytest
+import time_machine
+
+from documents.search._dates import _precision_bounds
+
+if TYPE_CHECKING:
+    import tantivy
+from documents.search._query import _FIELD_BOOSTS
+from documents.search._query import DEFAULT_SEARCH_FIELDS
+from documents.search._translate import OPEN_HI
+from documents.search._translate import OPEN_LO
+from documents.search._translate import Comma
+from documents.search._translate import FieldRange
+from documents.search._translate import FieldValue
+from documents.search._translate import FieldValueList
+from documents.search._translate import InvalidDateQuery
+from documents.search._translate import Passthrough
+from documents.search._translate import resolve_commas
+from documents.search._translate import scan
+from documents.search._translate import translate_query
+from documents.search._translate import translate_range
+from documents.search._translate import translate_scalar
+
+
+@pytest.mark.search
+class TestPrecisionBounds:
+    @pytest.mark.parametrize(
+        ("digits", "expected"),
+        [
+            ("2020", ((2020, 1, 1), (2021, 1, 1))),
+            ("202003", ((2020, 3, 1), (2020, 4, 1))),
+            ("202012", ((2020, 12, 1), (2021, 1, 1))),
+            ("20200115", ((2020, 1, 15), (2020, 1, 16))),
+            ("20201231", ((2020, 12, 31), (2021, 1, 1))),
+        ],
+    )
+    def test_valid(self, digits, expected):
+        lo, hi = _precision_bounds(digits)
+        assert (lo.year, lo.month, lo.day) == expected[0]
+        assert (hi.year, hi.month, hi.day) == expected[1]
+
+    @pytest.mark.parametrize("digits", ["202023", "20200230", "20201301", "20", "abcd"])
+    def test_invalid_returns_none(self, digits):
+        assert _precision_bounds(digits) is None
+
+
+@pytest.mark.search
+class TestScan:
+    def test_plain_words_are_passthrough(self):
+        assert scan("bank statement") == [Passthrough("bank statement")]
+
+    def test_field_value(self):
+        assert scan("created:2020") == [FieldValue("created", "2020")]
+
+    def test_field_value_in_boolean(self):
+        toks = scan("created:2020 OR foo")
+        assert toks == [
+            FieldValue("created", "2020"),
+            Passthrough(" OR foo"),
+        ]
+
+    def test_field_value_in_parens(self):
+        toks = scan("(created:2020 OR foo)")
+        assert toks == [
+            Passthrough("("),
+            FieldValue("created", "2020"),
+            Passthrough(" OR foo)"),
+        ]
+
+    def test_quoted_value(self):
+        assert scan('correspondent:"A B"') == [FieldValue("correspondent", '"A B"')]
+
+    def test_field_range(self):
+        assert scan("created:[2020 TO 2021]") == [
+            FieldRange("created", "[", "2020", "2021", "]"),
+        ]
+
+    @pytest.mark.parametrize(
+        ("query", "expected"),
+        [
+            pytest.param(
+                "created:[2020 to]",
+                FieldRange("created", "[", "2020", "", "]"),
+                id="open_upper",
+            ),
+            pytest.param(
+                "created:[to 2020]",
+                FieldRange("created", "[", "", "2020", "]"),
+                id="open_lower",
+            ),
+        ],
+    )
+    def test_open_range(self, query, expected):
+        assert scan(query) == [expected]
+
+    def test_comma_inside_range_not_split(self):
+        # No depth-0 comma here; the whole thing is one range token.
+        toks = scan("created:[2020 TO 2021]")
+        assert len(toks) == 1
+
+    # --- Edge-case / regression tests (scan must never raise) ---
+
+    def test_url_is_passthrough(self):
+        # "http" is not a known field; the whole URL must pass through verbatim.
+        assert scan("http://example.com") == [Passthrough("http://example.com")]
+
+    def test_unterminated_quote_is_passthrough(self):
+        # title is a known field but the quoted value has no closing quote;
+        # _consume_value returns None so the whole string falls into passthrough.
+        assert scan('title:"abc') == [Passthrough('title:"abc')]
+
+    def test_unterminated_bracket_is_passthrough(self):
+        # created is a known field but the range bracket is never closed;
+        # _consume_range returns None so the whole string falls into passthrough.
+        assert scan("created:[2020") == [Passthrough("created:[2020")]
+
+    def test_empty_value_at_end_is_passthrough(self):
+        # created is a known field but there is no value after the colon
+        # (_consume_value returns None for start >= n), so passthrough.
+        assert scan("created:") == [Passthrough("created:")]
+
+    def test_value_containing_colon(self):
+        # The bare-word value reader stops at whitespace/paren, not at colon,
+        # so "2020:30" is consumed as a single value token.
+        assert scan("created:2020:30") == [FieldValue("created", "2020:30")]
+
+    def test_comma_followed_by_unconsumable_value_stops(self):
+        # A comma followed by whitespace is neither a value-list continuation nor a
+        # clause separator: the value stops and the comma stays as passthrough.
+        assert scan("tag:foo, bar") == [
+            FieldValue("tag", "foo"),
+            Passthrough(", bar"),
+        ]
+
+    def test_bracket_without_to_is_open_upper_bound(self):
+        # A bracketed value with no TO falls back to (value, "") -> open upper bound.
+        assert scan("created:[2020]") == [
+            FieldRange("created", "[", "2020", "", "]"),
+        ]
+
+    def test_known_field_name_midword_is_passthrough(self):
+        # A known field name embedded mid-word is not a field token (the
+        # word-boundary guard); the whole run stays passthrough.
+        assert scan("xtag:foo") == [Passthrough("xtag:foo")]
+
+
+@pytest.mark.search
+class TestCommaResolution:
+    def test_value_list_multi_value_field(self):
+        toks = resolve_commas(scan("tag:foo,bar"))
+        assert toks == [FieldValueList("tag", ("foo", "bar"))]
+
+    def test_value_list_three(self):
+        toks = resolve_commas(scan("tag_id:1,2,3"))
+        assert toks == [FieldValueList("tag_id", ("1", "2", "3"))]
+
+    def test_text_field_comma_is_literal(self):
+        # correspondent is not multi-value: comma stays inside the value.
+        toks = resolve_commas(scan("correspondent:foo,bar"))
+        assert toks == [FieldValue("correspondent", "foo,bar")]
+
+    def test_clause_separator_before_known_field(self):
+        toks = resolve_commas(scan("tag:foo,type:bar"))
+        assert toks == [FieldValue("tag", "foo"), Comma(), FieldValue("type", "bar")]
+
+    def test_clause_separator_after_range(self):
+        toks = resolve_commas(scan("created:[2020 TO 2021],added:[2022 TO 2023]"))
+        assert toks == [
+            FieldRange("created", "[", "2020", "2021", "]"),
+            Comma(),
+            FieldRange("added", "[", "2022", "2023", "]"),
+        ]
+
+    def test_clause_separator_after_quote(self):
+        toks = resolve_commas(scan('correspondent:"A B",created:[2020 TO 2021]'))
+        assert toks == [
+            FieldValue("correspondent", '"A B"'),
+            Comma(),
+            FieldRange("created", "[", "2020", "2021", "]"),
+        ]
+
+    def test_url_comma_is_literal_passthrough(self):
+        toks = resolve_commas(scan("http://example.com/a,b"))
+        assert toks == [Passthrough("http://example.com/a,b")]
+
+    def test_non_multi_value_comma_is_literal(self):
+        # title is not in MULTI_VALUE_FIELDS: comma stays inside the value.
+        toks = resolve_commas(scan("title:10,20"))
+        assert toks == [FieldValue("title", "10,20")]
+
+    def test_clause_separator_before_known_date_field(self):
+        # The comma between a bare value and a known date field acts as a
+        # clause separator; both sides survive as distinct tokens.
+        toks = resolve_commas(scan("correspondent:foo,created:[2020 TO 2021]"))
+        assert toks == [
+            FieldValue("correspondent", "foo"),
+            Comma(),
+            FieldRange("created", "[", "2020", "2021", "]"),
+        ]
+
+
+@pytest.mark.search
+class TestTranslateScalar:
+    @pytest.mark.parametrize(
+        ("field", "value", "expected"),
+        [
+            (
+                "created",
+                "2020",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
+            ),
+            (
+                "created",
+                "202003",
+                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z}",
+            ),
+            (
+                "created",
+                "20200115",
+                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z}",
+            ),
+            (
+                "created",
+                "2020-01-15",
+                "created:[2020-01-15T00:00:00Z TO 2020-01-16T00:00:00Z}",
+            ),
+            (
+                "created",
+                "2020-03",
+                "created:[2020-03-01T00:00:00Z TO 2020-04-01T00:00:00Z}",
+            ),
+        ],
+    )
+    def test_partial_and_iso_dates(self, field: str, value: str, expected: str) -> None:
+        assert translate_scalar(field, value, UTC) == expected
+
+    def test_invalid_date_raises(self) -> None:
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_scalar("created", "202023", UTC)
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "202023"
+
+    def test_keyword_delegates(self) -> None:
+        # keyword path produces a half-open range; just assert it is a created range
+        out = translate_scalar("created", "today", UTC)
+        assert out.startswith("created:[") and out.endswith("}")
+
+    def test_14digit_compact_datetime(self) -> None:
+        out = translate_scalar("created", "20240115120000", UTC)
+        assert "20240115120000" not in out
+        assert out.startswith("created:")
+        assert out == "created:[2024-01-15T12:00:00Z TO 2024-01-15T12:00:00Z]"
+
+    def test_14digit_invalid_month_raises(self) -> None:
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_scalar("created", "20231300120000", UTC)
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "20231300120000"
+
+    def test_unrecognized_value_raises(self) -> None:
+        # A value that is not a keyword, digits, ISO date, or compact timestamp
+        # raises rather than producing invalid Tantivy syntax or silently matching
+        # nothing.
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_scalar("created", "garbage", UTC)
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "garbage"
+
+
+@pytest.mark.search
+class TestTranslateRange:
+    @pytest.mark.parametrize(
+        ("lo", "hi", "expected"),
+        [
+            ("2005", "2009", "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z}"),
+            (
+                "202001",
+                "202006",
+                "created:[2020-01-01T00:00:00Z TO 2020-07-01T00:00:00Z}",
+            ),
+            (
+                "20200101",
+                "20201231",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
+            ),
+            (
+                "2020-01-01",
+                "2020-12-31",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
+            ),
+        ],
+    )
+    def test_absolute_ranges(self, lo, hi, expected):
+        assert translate_range("created", lo, hi, UTC) == expected
+
+    def test_reversed_swaps(self):
+        assert translate_range("created", "2009", "2005", UTC) == (
+            "created:[2005-01-01T00:00:00Z TO 2010-01-01T00:00:00Z}"
+        )
+
+    def test_open_upper(self):
+        out = translate_range("created", "2020", "", UTC)
+        assert out == f"created:[2020-01-01T00:00:00Z TO {OPEN_HI}]"
+
+    def test_open_lower(self):
+        out = translate_range("created", "", "2020", UTC)
+        assert out == f"created:[{OPEN_LO} TO 2021-01-01T00:00:00Z}}"
+
+    def test_invalid_bound_raises(self):
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_range("created", "202023", "2025", UTC)
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "202023"
+
+    def test_invalid_high_bound_raises(self):
+        # Low bound parses, high bound does not -> raise on the high bound.
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_range("created", "2020", "garbage", UTC)
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "garbage"
+
+
+@pytest.mark.search
+class TestTranslateQuery:
+    @pytest.mark.parametrize(
+        ("raw", "expected"),
+        [
+            (
+                "created:2020",
+                "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z}",
+            ),
+            ("tag:foo,bar", "tag:foo AND tag:bar"),
+            # 'type' is a user-facing alias rewritten to 'document_type' (the real schema field)
+            ("tag:foo,type:bar", "tag:foo AND document_type:bar"),
+            (
+                "created:[2020 TO 2021],added:[2022 TO 2023]",
+                (
+                    "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z}"
+                    " AND "
+                    "added:[2022-01-01T00:00:00Z TO 2024-01-01T00:00:00Z}"
+                ),
+            ),
+            # correspondent is not multi-value: comma stays literal inside the value
+            ("correspondent:foo,bar", "correspondent:foo,bar"),
+        ],
+    )
+    def test_golden(self, raw: str, expected: str) -> None:
+        assert translate_query(raw, UTC) == expected
+
+    @pytest.mark.parametrize(
+        "raw",
+        [
+            "created:2020",
+            "created:202003",
+            "created:[20200101 TO 20201231]",
+            "created:[2020-01-01 TO 2020-12-31]",
+            "created:[2020 to]",
+            "created:[to 2020]",
+            "title:x,created:[2020 TO 2021]",
+            "created:2020 OR foo",
+            "(created:2020 OR invoice)",
+            "tag:foo,type:bar",
+            "bank statement",
+        ],
+    )
+    def test_parse_acceptance(self, index: tantivy.Index, raw: str) -> None:
+        translated = translate_query(raw, UTC)
+        # Must not raise:
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+
+@pytest.mark.search
+class TestFieldAliasing:
+    """Whoosh->Tantivy field-name aliasing (type/path -> document_type/storage_path)."""
+
+    def test_type_alias(self) -> None:
+        assert translate_query("type:invoice", UTC) == "document_type:invoice"
+
+    def test_path_alias(self) -> None:
+        assert translate_query("path:/foo/bar", UTC) == "storage_path:/foo/bar"
+
+    def test_type_id_alias(self) -> None:
+        assert translate_query("type_id:5", UTC) == "document_type_id:5"
+
+    def test_path_id_alias(self) -> None:
+        assert translate_query("path_id:7", UTC) == "storage_path_id:7"
+
+    def test_clause_separator_plus_alias(self) -> None:
+        # Comma between known fields acts as AND separator; alias still applied.
+        assert (
+            translate_query("tag:foo,type:bar", UTC) == "tag:foo AND document_type:bar"
+        )
+
+    def test_type_range_alias(self) -> None:
+        # type is not a date field; range passes through verbatim with alias applied.
+        assert (
+            translate_query("type:[2020 TO 2021]", UTC)
+            == "document_type:[2020 TO 2021]"
+        )
+
+    def test_parse_acceptance_type(self, index: tantivy.Index) -> None:
+        # Translated output must be accepted by the real Tantivy parser.
+        translated = translate_query("type:invoice", UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+    def test_parse_acceptance_path(self, index: tantivy.Index) -> None:
+        translated = translate_query("path:foo", UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+
+# Freeze time so relative-date tests are deterministic.
+_FROZEN_NOW = datetime(2026, 3, 28, 12, 0, 0, tzinfo=UTC)
+
+
+@pytest.mark.search
+class TestRelativeRanges:
+    """Relative date-range tokens resolved against a frozen clock."""
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_7_days_to_now(self) -> None:
+        assert translate_query("added:[-7 days to now]", UTC) == (
+            "added:[2026-03-21T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_1_week_to_now(self) -> None:
+        assert translate_query("added:[-1 week to now]", UTC) == (
+            "added:[2026-03-21T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_1_month_to_now(self) -> None:
+        assert translate_query("created:[-1 month to now]", UTC) == (
+            "created:[2026-02-28T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_1_year_to_now(self) -> None:
+        assert translate_query("modified:[-1 year to now]", UTC) == (
+            "modified:[2025-03-28T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_3_hours_to_now(self) -> None:
+        assert translate_query("added:[-3 hours to now]", UTC) == (
+            "added:[2026-03-28T09:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_uppercase_units(self) -> None:
+        assert translate_query("added:[-1 WEEK TO NOW]", UTC) == (
+            "added:[2026-03-21T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_now_minus_7d_compact(self) -> None:
+        assert translate_query("added:[now-7d TO now]", UTC) == (
+            "added:[2026-03-21T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_reversed_range_swapped(self) -> None:
+        # now+1h TO now-1h is reversed; translate_range swaps -> lo=now-1h, hi=now+1h
+        assert translate_query("added:[now+1h TO now-1h]", UTC) == (
+            "added:[2026-03-28T11:00:00Z TO 2026-03-28T13:00:00Z]"
+        )
+
+    @pytest.mark.parametrize(
+        "raw",
+        [
+            "added:[-7 days to now]",
+            "added:[-1 week to now]",
+            "created:[-1 month to now]",
+            "modified:[-1 year to now]",
+            "added:[-3 hours to now]",
+            "added:[now-7d TO now]",
+            "added:[now+1h TO now-1h]",
+        ],
+    )
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_parse_acceptance(self, index: tantivy.Index, raw: str) -> None:
+        translated = translate_query(raw, UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+
+@pytest.mark.search
+class TestWhooshUnitAbbreviations:
+    """
+    Whoosh's PlusMinus date grammar accepted abbreviated unit spellings
+    (e.g. "yrs", "mos", "wks", "hrs", "mins", "secs"); saved views/searches
+    created under the old Whoosh backend can contain those tokens (see
+    https://github.com/paperless-ngx/paperless-ngx/issues/13482), so the
+    Tantivy translator must still accept them.
+    """
+
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_minus_999_yrs(self) -> None:
+        assert translate_query("created:[-999yrs to now]", UTC) == (
+            "created:[1027-03-28T12:00:00Z TO 2026-03-28T12:00:00Z]"
+        )
+
+    @pytest.mark.parametrize(
+        ("token", "expected_lo"),
+        [
+            ("-1y", "2025-03-28T12:00:00Z"),
+            ("-1yr", "2025-03-28T12:00:00Z"),
+            ("-3mos", "2025-12-28T12:00:00Z"),
+            ("-3mo", "2025-12-28T12:00:00Z"),
+            ("-2wks", "2026-03-14T12:00:00Z"),
+            ("-2wk", "2026-03-14T12:00:00Z"),
+            ("-5dys", "2026-03-23T12:00:00Z"),
+            ("-5dy", "2026-03-23T12:00:00Z"),
+            ("-1hrs", "2026-03-28T11:00:00Z"),
+            ("-1hr", "2026-03-28T11:00:00Z"),
+            ("-10mins", "2026-03-28T11:50:00Z"),
+            ("-10min", "2026-03-28T11:50:00Z"),
+            ("-30secs", "2026-03-28T11:59:30Z"),
+            ("-30sec", "2026-03-28T11:59:30Z"),
+        ],
+    )
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_abbreviated_units(self, token: str, expected_lo: str) -> None:
+        assert translate_query(f"added:[{token} to now]", UTC) == (
+            f"added:[{expected_lo} TO 2026-03-28T12:00:00Z]"
+        )
+
+    @pytest.mark.parametrize(
+        "raw",
+        [
+            "created:[-999yrs to now]",
+            "added:[-1y to now]",
+            "created:[-3mos to now]",
+            "added:[-2wks to now]",
+            "added:[-5dys to now]",
+            "added:[-1hrs to now]",
+            "added:[-10mins to now]",
+            "added:[-30secs to now]",
+        ],
+    )
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_parse_acceptance(self, index: tantivy.Index, raw: str) -> None:
+        translated = translate_query(raw, UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+
+@pytest.mark.search
+class TestOperatorNormalization:
+    """Post-render operator normalization in translate_query."""
+
+    def test_spaced_dash_removed(self) -> None:
+        assert (
+            translate_query("H52.1 - Kurzsichtigkeit", UTC) == "H52.1 Kurzsichtigkeit"
+        )
+
+    def test_spaced_dash_simple(self) -> None:
+        assert translate_query("bar - baz", UTC) == "bar baz"
+
+    def test_trailing_operator_stripped(self) -> None:
+        assert translate_query("foo -", UTC) == "foo"
+
+    def test_date_range_preserved(self) -> None:
+        out = translate_query("created:[2020 TO 2021]", UTC)
+        # Must not corrupt the ISO range
+        assert out == "created:[2020-01-01T00:00:00Z TO 2022-01-01T00:00:00Z}"
+
+    def test_date_scalar_with_or(self) -> None:
+        out = translate_query("created:2020 OR foo", UTC)
+        # The created scalar becomes a range; " OR foo" passes through verbatim.
+        assert out.startswith("created:[")
+        assert "OR foo" in out
+
+    def test_parse_acceptance_spaced_dash(self, index: tantivy.Index) -> None:
+        translated = translate_query("H52.1 - Kurzsichtigkeit", UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+    def test_parse_acceptance_trailing_op(self, index: tantivy.Index) -> None:
+        translated = translate_query("foo -", UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+
+@pytest.mark.search
+class TestMultiWordDateKeywords:
+    """scan() must consume multi-word date keywords as a single value."""
+
+    def test_scan_previous_week_as_single_token(self) -> None:
+        # "created:previous week" must produce one FieldValue with value "previous week",
+        # not FieldValue("created","previous") + Passthrough(" week").
+        toks = scan("created:previous week")
+        assert toks == [FieldValue("created", "previous week")]
+
+    def test_scan_this_month_as_single_token(self) -> None:
+        toks = scan("added:this month")
+        assert toks == [FieldValue("added", "this month")]
+
+    def test_scan_previous_month_as_single_token(self) -> None:
+        toks = scan("created:previous month")
+        assert toks == [FieldValue("created", "previous month")]
+
+    def test_scan_this_year_as_single_token(self) -> None:
+        toks = scan("added:this year")
+        assert toks == [FieldValue("added", "this year")]
+
+    def test_scan_previous_year_as_single_token(self) -> None:
+        toks = scan("created:previous year")
+        assert toks == [FieldValue("created", "previous year")]
+
+    def test_scan_previous_quarter_as_single_token(self) -> None:
+        toks = scan("created:previous quarter")
+        assert toks == [FieldValue("created", "previous quarter")]
+
+    def test_quoted_multi_word_keyword_still_works(self) -> None:
+        # The quoted form must continue to work as before.
+        toks = scan('created:"previous week"')
+        assert toks == [FieldValue("created", '"previous week"')]
+
+    def test_non_date_field_not_affected(self) -> None:
+        # "previous" stops at the space for non-date fields; " week" passes through.
+        toks = scan("correspondent:previous week")
+        assert toks == [
+            FieldValue("correspondent", "previous"),
+            Passthrough(" week"),
+        ]
+
+
+@pytest.mark.search
+class TestKeywordDateResolution:
+    """Relative date keywords resolve to exact ISO ranges against a frozen clock.
+
+    Frozen at 2026-03-28 12:00 UTC (a Saturday in Q1) so the week, month,
+    quarter and year rollovers are all exercised by a single anchor.
+    """
+
+    # created is a DateField: bounds are UTC midnight, no timezone offset.
+    @pytest.mark.parametrize(
+        ("keyword", "expected"),
+        [
+            pytest.param(
+                "today",
+                "created:[2026-03-28T00:00:00Z TO 2026-03-29T00:00:00Z}",
+                id="today",
+            ),
+            pytest.param(
+                "yesterday",
+                "created:[2026-03-27T00:00:00Z TO 2026-03-28T00:00:00Z}",
+                id="yesterday",
+            ),
+            pytest.param(
+                "previous week",
+                "created:[2026-03-16T00:00:00Z TO 2026-03-23T00:00:00Z}",
+                id="previous-week",
+            ),
+            pytest.param(
+                "this month",
+                "created:[2026-03-01T00:00:00Z TO 2026-04-01T00:00:00Z}",
+                id="this-month",
+            ),
+            pytest.param(
+                "previous month",
+                "created:[2026-02-01T00:00:00Z TO 2026-03-01T00:00:00Z}",
+                id="previous-month",
+            ),
+            pytest.param(
+                "this year",
+                "created:[2026-01-01T00:00:00Z TO 2027-01-01T00:00:00Z}",
+                id="this-year",
+            ),
+            pytest.param(
+                "previous year",
+                "created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z}",
+                id="previous-year",
+            ),
+            pytest.param(
+                "previous quarter",
+                "created:[2025-10-01T00:00:00Z TO 2026-01-01T00:00:00Z}",
+                id="previous-quarter",
+            ),
+        ],
+    )
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_date_only_field_keyword_ranges(
+        self,
+        keyword: str,
+        expected: str,
+    ) -> None:
+        assert translate_query(f"created:{keyword}", UTC) == expected
+
+    # added is a DateTimeField: local-tz midnight converted to UTC. Tokyo
+    # (+09:00, no DST) shifts each midnight boundary back to 15:00Z the day
+    # before, so this also exercises the local-midnight offset path.
+    @pytest.mark.parametrize(
+        ("keyword", "expected"),
+        [
+            pytest.param(
+                "today",
+                "added:[2026-03-27T15:00:00Z TO 2026-03-28T15:00:00Z}",
+                id="today",
+            ),
+            pytest.param(
+                "yesterday",
+                "added:[2026-03-26T15:00:00Z TO 2026-03-27T15:00:00Z}",
+                id="yesterday",
+            ),
+            pytest.param(
+                "previous week",
+                "added:[2026-03-15T15:00:00Z TO 2026-03-22T15:00:00Z}",
+                id="previous-week",
+            ),
+            pytest.param(
+                "this month",
+                "added:[2026-02-28T15:00:00Z TO 2026-03-31T15:00:00Z}",
+                id="this-month",
+            ),
+            pytest.param(
+                "previous month",
+                "added:[2026-01-31T15:00:00Z TO 2026-02-28T15:00:00Z}",
+                id="previous-month",
+            ),
+            pytest.param(
+                "this year",
+                "added:[2025-12-31T15:00:00Z TO 2026-12-31T15:00:00Z}",
+                id="this-year",
+            ),
+            pytest.param(
+                "previous year",
+                "added:[2024-12-31T15:00:00Z TO 2025-12-31T15:00:00Z}",
+                id="previous-year",
+            ),
+            pytest.param(
+                "previous quarter",
+                "added:[2025-09-30T15:00:00Z TO 2025-12-31T15:00:00Z}",
+                id="previous-quarter",
+            ),
+        ],
+    )
+    @time_machine.travel(_FROZEN_NOW, tick=False)
+    def test_datetime_field_keyword_ranges_local_tz(
+        self,
+        keyword: str,
+        expected: str,
+    ) -> None:
+        assert translate_query(f"added:{keyword}", ZoneInfo("Asia/Tokyo")) == expected
+
+
+@pytest.mark.search
+class TestISODatetimeBounds:
+    """Full ISO datetime tokens in range bounds must be parsed directly."""
+
+    def test_translate_range_iso_bounds_passthrough(self) -> None:
+        # Already-ISO datetime bounds must pass through as-is (exact instant).
+        result = translate_range(
+            "created",
+            "2020-01-01T00:00:00Z",
+            "2021-01-01T00:00:00Z",
+            UTC,
+        )
+        assert result == "created:[2020-01-01T00:00:00Z TO 2021-01-01T00:00:00Z]"
+
+    def test_translate_query_iso_range_preserved(self) -> None:
+        q = "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        assert translate_query(q, UTC) == q
+
+    def test_translate_query_comma_separated_iso_ranges(self) -> None:
+        q = (
+            "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z],"
+            "added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        )
+        result = translate_query(q, UTC)
+        assert result == (
+            "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+            " AND "
+            "added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        )
+
+    def test_translate_query_text_before_comma_separated_date_clause(self) -> None:
+        result = translate_query("schäfersee,created:previous year", UTC)
+        assert result == (
+            "schäfersee AND created:[2025-01-01T00:00:00Z TO 2026-01-01T00:00:00Z}"
+        )
+
+    def test_invalid_iso_datetime_raises(self) -> None:
+        # A token with "T" that is not valid ISO datetime -> raise.
+        with pytest.raises(InvalidDateQuery) as exc_info:
+            translate_range(
+                "created",
+                "2020-01-01T99:00:00Z",
+                "2021-01-01T00:00:00Z",
+                UTC,
+            )
+        assert exc_info.value.field == "created"
+        assert exc_info.value.value == "2020-01-01T99:00:00Z"
+
+    def test_parse_acceptance_iso_bounds(self, index: tantivy.Index) -> None:
+        q = "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        translated = translate_query(q, UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)
+
+    def test_parse_acceptance_comma_iso_ranges(self, index: tantivy.Index) -> None:
+        q = (
+            "created:[2026-01-01T00:00:00Z TO 2026-06-01T00:00:00Z],"
+            "added:[2026-05-01T00:00:00Z TO 2026-06-01T00:00:00Z]"
+        )
+        translated = translate_query(q, UTC)
+        index.parse_query(translated, DEFAULT_SEARCH_FIELDS, field_boosts=_FIELD_BOOSTS)

src/documents/tests/test_api_app_config.py

@@ -72,6 +72,10 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
                 "barcode_enable_tag": None,
                 "barcode_tag_mapping": None,
                 "barcode_tag_split": None,
+                "remote_ocr_engine": None,
+                "remote_ocr_api_key": None,
+                "remote_ocr_endpoint": None,
+                "remote_ocr_mode": None,
                 "ai_enabled": False,
                 "llm_embedding_backend": None,
                 "llm_embedding_model": None,
@@ -870,6 +874,49 @@ class TestApiAppConfig(DirectoriesMixin, APITestCase):
         config.refresh_from_db()
         self.assertEqual(config.llm_api_key, None)
 
+    def test_update_remote_ocr_api_key(self) -> None:
+        """
+        GIVEN:
+            - Existing config with remote_ocr_api_key specified
+        WHEN:
+            - API to update remote_ocr_api_key is called with all *s
+            - API to update remote_ocr_api_key is called with empty string
+        THEN:
+            - remote_ocr_api_key is unchanged
+            - remote_ocr_api_key is set to None
+        """
+        config = ApplicationConfiguration.objects.first()
+        assert config is not None
+        config.remote_ocr_api_key = "1234567890"
+        config.save()
+
+        # Test with all *
+        response = self.client.patch(
+            f"{self.ENDPOINT}1/",
+            json.dumps(
+                {
+                    "remote_ocr_api_key": "*" * 32,
+                },
+            ),
+            content_type="application/json",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        config.refresh_from_db()
+        self.assertEqual(config.remote_ocr_api_key, "1234567890")
+        # Test with empty string
+        response = self.client.patch(
+            f"{self.ENDPOINT}1/",
+            json.dumps(
+                {
+                    "remote_ocr_api_key": "",
+                },
+            ),
+            content_type="application/json",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        config.refresh_from_db()
+        self.assertEqual(config.remote_ocr_api_key, None)
+
     def test_enable_ai_index_triggers_update(self) -> None:
         """
         GIVEN:

src/documents/tests/test_api_bulk_download.py

@@ -339,21 +339,3 @@ class TestBulkDownload(DirectoriesMixin, SampleDirMixin, APITestCase):
 
         self.assertEqual(response.status_code, status.HTTP_403_FORBIDDEN)
         self.assertEqual(response.content, b"Insufficient permissions")
-
-    def test_bad_search_query_returns_400(self) -> None:
-        response = self.client.post(
-            self.ENDPOINT,
-            json.dumps(
-                {
-                    "all": True,
-                    "filters": {"query": "added:notadate"},
-                    "content": "originals",
-                },
-            ),
-            content_type="application/json",
-        )
-
-        # A user-fixable query error must surface as a 400 naming the bad
-        # value, exactly like the search list endpoint, never a 500.
-        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
-        self.assertIn(b"notadate", response.content)

src/documents/tests/test_api_bulk_edit.py

@@ -532,7 +532,29 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
         m.assert_called_once()
         args, kwargs = m.call_args
         self.assertEqual(args[0], [self.doc1.id])
-        self.assertEqual(len(kwargs), 0)
+        self.assertEqual(kwargs, {"remote_ocr": False})
+
+    @mock.patch("documents.views.bulk_edit.reprocess")
+    def test_reprocess_documents_endpoint_remote_ocr(self, m) -> None:
+        """
+        GIVEN:
+            - API data to reprocess a document with remote OCR requested
+        WHEN:
+            - API is called
+        THEN:
+            - reprocess is called with remote_ocr=True
+        """
+        self.setup_mock(m, "reprocess")
+        response = self.client.post(
+            "/api/documents/reprocess/",
+            json.dumps({"documents": [self.doc1.id], "remote_ocr": True}),
+            content_type="application/json",
+        )
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        m.assert_called_once()
+        args, kwargs = m.call_args
+        self.assertEqual(args[0], [self.doc1.id])
+        self.assertEqual(kwargs, {"remote_ocr": True})
 
     @mock.patch("documents.serialisers.bulk_edit.set_storage_path")
     def test_api_set_storage_path(self, m) -> None:
@@ -1553,6 +1575,29 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
                             ),
                         )
 
+    def test_legacy_bulk_edit_reprocess_invalid_remote_ocr(self) -> None:
+        """
+        GIVEN:
+            - The deprecated bulk_edit endpoint with a non-boolean remote_ocr
+        WHEN:
+            - API is called
+        THEN:
+            - The request is rejected rather than passed through to the task
+        """
+        response = self.client.post(
+            "/api/documents/bulk_edit/",
+            json.dumps(
+                {
+                    "documents": [self.doc1.id],
+                    "method": "reprocess",
+                    "parameters": {"remote_ocr": "yes please"},
+                },
+            ),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+
     @mock.patch("documents.views.bulk_edit.edit_pdf")
     def test_edit_pdf(self, m) -> None:
         self.setup_mock(m, "edit_pdf")
@@ -1976,22 +2021,3 @@ class TestBulkEditAPI(DirectoriesMixin, APITestCase):
 
         self.assertEqual(response.status_code, status.HTTP_200_OK)
         self.assertEqual(LogEntry.objects.filter(object_pk=self.doc1.id).count(), 2)
-
-    def test_api_bulk_edit_with_bad_search_query_returns_400(self) -> None:
-        response = self.client.post(
-            "/api/documents/bulk_edit/",
-            json.dumps(
-                {
-                    "all": True,
-                    "filters": {"query": "added:notadate"},
-                    "method": "set_storage_path",
-                    "parameters": {"storage_path": self.sp1.id},
-                },
-            ),
-            content_type="application/json",
-        )
-
-        # A user-fixable query error must surface as a 400 naming the bad
-        # value, exactly like the search list endpoint, never a 500.
-        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
-        self.assertIn(b"notadate", response.content)

src/documents/tests/test_api_document_versions.py

@@ -11,6 +11,7 @@ from django.contrib.auth.models import User
 from django.contrib.contenttypes.models import ContentType
 from django.core.exceptions import FieldError
 from django.core.files.uploadedfile import SimpleUploadedFile
+from django.test import TestCase as DjangoTestCase
 from django.utils import timezone
 from rest_framework import status
 from rest_framework.test import APITestCase
@@ -21,6 +22,7 @@ from documents.filters import TitleContentFilter
 from documents.models import Document
 from documents.tests.utils import DirectoriesMixin
 from documents.tests.utils import read_streaming_response
+from documents.views import DocumentSelectionMixin
 
 if TYPE_CHECKING:
     from pathlib import Path
@@ -923,3 +925,36 @@ class TestVersionAwareFilters(TestCase):
 
         self.assertIs(result, queryset)
         queryset.filter.assert_not_called()
+
+
+class TestBulkSelectionExcludesVersions(DjangoTestCase):
+    def test_select_all_matching_does_not_select_version_documents(self) -> None:
+        """
+        "Select all matching" reconstructs the document list, which never
+        contains version documents as rows of their own.
+        """
+        user = User.objects.create_superuser(username="bulk_versions")
+        root = Document.objects.create(
+            title="shared-title root",
+            checksum="bulk-root",
+            mime_type="application/pdf",
+            content="root",
+        )
+        Document.objects.create(
+            title="shared-title version",
+            checksum="bulk-version",
+            mime_type="application/pdf",
+            root_document=root,
+            version_index=1,
+            content="version",
+        )
+
+        selected = DocumentSelectionMixin()._resolve_document_ids(
+            user=user,
+            validated_data={
+                "all": True,
+                "filters": {"title__icontains": "shared-title"},
+            },
+        )
+
+        self.assertEqual(selected, [root.id])

src/documents/tests/test_api_documents.py

@@ -2486,7 +2486,7 @@ class TestDocumentApi(DirectoriesMixin, ConsumeTaskMixin, APITestCase):
         response = self.client.get("/api/documents/34676/suggestions/")
         self.assertEqual(response.status_code, status.HTTP_404_NOT_FOUND)
 
-    @mock.patch("documents.views.get_ai_document_classification")
+    @mock.patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(AI_ENABLED=True)
     def test_suggestions_still_uses_classifier_when_ai_enabled(
         self,

src/documents/tests/test_api_search.py

@@ -756,10 +756,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
             tick=False,
         ):
             response = self.client.get("/api/documents/?query=added:previous month")
-        assert response.status_code == 200, (
-            f"expected a successful search response, got {response.status_code}: "
-            f"{response.data!r}"
-        )
         results = response.data["results"]
 
         self.assertEqual(len(results), 1)
@@ -792,26 +788,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
         self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
         self.assertIn("invalid-date", str(response.data["query"]))
 
-    def test_search_multiple_bad_fields_returns_all_messages(self) -> None:
-        """
-        GIVEN:
-            - One document added
-        WHEN:
-            - Query with multiple bad fields (e.g. invalid date and invalid number)
-        THEN:
-            - 400 Bad Request with error messages for every bad field,
-              so the user can fix them all in one round-trip
-        """
-        response = self.client.get(
-            "/api/documents/",
-            {"query": "created:notadate AND asn:notanumber"},
-        )
-        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
-        messages = response.data["query"]
-        self.assertEqual(len(messages), 2)
-        self.assertTrue(any("created" in m for m in messages))
-        self.assertTrue(any("asn" in m for m in messages))
-
     @override_settings(
         TIME_ZONE="UTC",
     )
@@ -855,29 +831,6 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
         results = response.data["results"]
         self.assertEqual({r["id"] for r in results}, {1, 2})
 
-    @mock.patch("documents.search._backend.parse_user_query")
-    def test_search_parser_bug_surfaces_as_500_not_400(self, m) -> None:
-        """
-        GIVEN:
-            - The query parser itself fails (a whoosh-compat bug, per
-              QueryParserError's own contract: not user-fixable input)
-        WHEN:
-            - Any search request runs
-        THEN:
-            - The error surfaces as a 500 monitoring can see, never a 400
-              blaming the user for a library defect
-        """
-        from whoosh_compat.errors import QueryParserError
-
-        m.side_effect = QueryParserError("synthetic parser bug")
-
-        self.client.raise_request_exception = False
-        response = self.client.get("/api/documents/?query=anything")
-        self.assertEqual(
-            response.status_code,
-            status.HTTP_500_INTERNAL_SERVER_ERROR,
-        )
-
     @mock.patch("documents.search._backend.TantivyBackend.autocomplete")
     def test_search_autocomplete_limits(self, m) -> None:
         """
@@ -2052,45 +2005,3 @@ class TestDocumentSearchApi(DirectoriesMixin, APITestCase):
         self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
         response = self.client.get("/api/search/?query=no")
         self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
-
-    def _assert_query_finds(self, doc: Document, query: str) -> None:
-        get_backend().add_or_update(doc)
-        response = self.client.get("/api/documents/", {"query": query})
-        self.assertEqual(response.status_code, status.HTTP_200_OK)
-        ids = [r["id"] for r in response.data["results"]]
-        self.assertIn(doc.id, ids)
-
-    def test_search_by_asn(self) -> None:
-        doc = Document.objects.create(
-            title="Has ASN",
-            content="content",
-            checksum="asn-checksum",
-            archive_serial_number=555,
-        )
-        self._assert_query_finds(doc, "asn:555")
-
-    def test_search_by_page_count(self) -> None:
-        doc = Document.objects.create(
-            title="Multi-page",
-            content="content",
-            checksum="page-count-checksum",
-            page_count=42,
-        )
-        self._assert_query_finds(doc, "page_count:42")
-
-    def test_search_by_original_filename(self) -> None:
-        doc = Document.objects.create(
-            title="Named file",
-            content="content",
-            checksum="filename-checksum",
-            original_filename="quarterly-report.pdf",
-        )
-        self._assert_query_finds(doc, "original_filename:quarterly-report.pdf")
-
-    def test_search_by_checksum(self) -> None:
-        doc = Document.objects.create(
-            title="Checksum doc",
-            content="content",
-            checksum="deadbeef1234",
-        )
-        self._assert_query_finds(doc, "checksum:deadbeef1234")

src/documents/tests/test_api_search_errors.py

@@ -1,275 +0,0 @@
-"""The search list endpoint's exception handling: what becomes a 400 and
-what a library defect surfaces as instead.
-
-Companion to documents/tests/search/test_error_routing.py, which pins the
-Cause -> SearchQueryError/QueryError routing inside documents/search/_query.py.
-These tests pin the layer above it: DocumentViewSet.list's own except clauses,
-which decide what an already-routed error becomes on the wire.
-"""
-
-from __future__ import annotations
-
-from typing import TYPE_CHECKING
-
-import pytest
-from rest_framework import status
-from whoosh_compat.errors import Cause
-from whoosh_compat.errors import Diagnostic
-from whoosh_compat.errors import DiagnosticKind
-from whoosh_compat.errors import QueryError
-
-from documents.search import SearchQueryError
-from documents.tests.factories import DocumentFactory
-
-if TYPE_CHECKING:
-    from rest_framework.test import APIClient
-
-    from documents.models import Document
-
-pytestmark = [pytest.mark.django_db, pytest.mark.usefixtures("_search_index")]
-
-
-@pytest.fixture
-def indexed_document() -> Document:
-    from documents.search import get_backend
-
-    doc = DocumentFactory.create(title="quarterly invoice", content="acme corp")
-    get_backend().add_or_update(doc)
-    return doc
-
-
-class TestSearchQueryErrorStillBecomesA400:
-    def test_search_query_error_becomes_a_400_naming_the_field(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_search_query_error(*args: object, **kwargs: object) -> object:
-            raise SearchQueryError("bad value for field 'added'")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_user_query",
-            raise_search_query_error,
-        )
-
-        response = admin_client.get("/api/documents/?query=anything")
-
-        assert response.status_code == status.HTTP_400_BAD_REQUEST
-        assert "added" in str(response.data["query"])
-
-
-class TestLibraryDefectsPropagate:
-    """The exact regression this task exists to fix: an unexpected or
-    INTERNAL-cause library error must not be relabeled a 400."""
-
-    def test_unexpected_exception_is_not_converted_to_a_400(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_zero_division(*args: object, **kwargs: object) -> object:
-            raise ZeroDivisionError("synthetic bug, unrelated to search grammar")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_user_query",
-            raise_zero_division,
-        )
-
-        with pytest.raises(ZeroDivisionError):
-            admin_client.get("/api/documents/?query=anything")
-
-    def test_internal_cause_query_error_is_not_converted_to_a_400(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        """Forces the one library-internal failure mode reachable from a real
-        query: emit() reporting a defect in itself (Cause.INTERNAL) after a
-        real query string went through the real parse and routing pipeline.
-
-        ``tantivy_emit`` (the whoosh-compat emitter) is monkeypatched rather
-        than ``parse_user_query`` itself, so everything upstream of it --
-        the pre-parse rewrites, ``wc.parse()``, and ``_map_emit_error``'s own
-        Cause routing in documents/search/_query.py -- runs for real; only
-        the final emit call is forced to report the defect.
-        """
-        import documents.search._query as query_mod
-
-        def raise_internal(*args: object, **kwargs: object) -> object:
-            raise QueryError(
-                Diagnostic(
-                    kind=DiagnosticKind.BACKEND_REJECTED,
-                    cause=Cause.INTERNAL,
-                    message="synthetic whoosh-compat emitter defect",
-                ),
-            )
-
-        monkeypatch.setattr(query_mod, "tantivy_emit", raise_internal)
-
-        with pytest.raises(QueryError):
-            admin_client.get("/api/documents/?query=invoice")
-
-
-class TestSelectionPathsAgreeWithSearch:
-    """DocumentSelectionMixin backs bulk edit, bulk download, and a
-    more_like_id selection filter. It catches only SearchQueryError -- the
-    same contract the search list endpoint enforces above -- so all three
-    must map SearchQueryError to a 400 and let anything else surface."""
-
-    def test_bulk_edit_maps_search_query_error_to_a_400(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_search_query_error(*args: object, **kwargs: object) -> object:
-            raise SearchQueryError("bad value for field 'added'")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_user_query",
-            raise_search_query_error,
-        )
-
-        response = admin_client.post(
-            "/api/documents/bulk_edit/",
-            {
-                "documents": [],
-                "all": True,
-                "filters": {"query": "anything"},
-                "method": "set_document_type",
-                "parameters": {"document_type": None},
-            },
-            format="json",
-        )
-
-        assert response.status_code == status.HTTP_400_BAD_REQUEST
-        assert "added" in str(response.data["query"])
-
-    def test_bulk_edit_lets_an_unexpected_exception_surface(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_zero_division(*args: object, **kwargs: object) -> object:
-            raise ZeroDivisionError("synthetic bug, unrelated to search grammar")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_user_query",
-            raise_zero_division,
-        )
-
-        with pytest.raises(ZeroDivisionError):
-            admin_client.post(
-                "/api/documents/bulk_edit/",
-                {
-                    "documents": [],
-                    "all": True,
-                    "filters": {"query": "anything"},
-                    "method": "set_document_type",
-                    "parameters": {"document_type": None},
-                },
-                format="json",
-            )
-
-    def test_bulk_download_maps_search_query_error_to_a_400(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_search_query_error(*args: object, **kwargs: object) -> object:
-            raise SearchQueryError("bad value for field 'added'")
-
-        monkeypatch.setattr(
-            backend_mod,
-            "parse_user_query",
-            raise_search_query_error,
-        )
-
-        response = admin_client.post(
-            "/api/documents/bulk_download/",
-            {
-                "documents": [],
-                "all": True,
-                "filters": {"query": "anything"},
-            },
-            format="json",
-        )
-
-        assert response.status_code == status.HTTP_400_BAD_REQUEST
-        assert "added" in str(response.data["query"])
-
-    def test_more_like_id_selection_filter_maps_search_query_error_to_a_400(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_search_query_error(*args: object, **kwargs: object) -> object:
-            raise SearchQueryError("similar-document lookup is unavailable")
-
-        monkeypatch.setattr(
-            backend_mod.TantivyBackend,
-            "more_like_this_ids",
-            raise_search_query_error,
-        )
-
-        response = admin_client.post(
-            "/api/documents/bulk_download/",
-            {
-                "documents": [],
-                "all": True,
-                "filters": {"more_like_id": indexed_document.pk},
-            },
-            format="json",
-        )
-
-        assert response.status_code == status.HTTP_400_BAD_REQUEST
-
-    def test_more_like_id_selection_filter_lets_an_unexpected_exception_surface(
-        self,
-        admin_client: APIClient,
-        monkeypatch: pytest.MonkeyPatch,
-        indexed_document: Document,
-    ) -> None:
-        import documents.search._backend as backend_mod
-
-        def raise_zero_division(*args: object, **kwargs: object) -> object:
-            raise ZeroDivisionError("synthetic bug, unrelated to similarity lookup")
-
-        monkeypatch.setattr(
-            backend_mod.TantivyBackend,
-            "more_like_this_ids",
-            raise_zero_division,
-        )
-
-        with pytest.raises(ZeroDivisionError):
-            admin_client.post(
-                "/api/documents/bulk_download/",
-                {
-                    "documents": [],
-                    "all": True,
-                    "filters": {"more_like_id": indexed_document.pk},
-                },
-                format="json",
-            )

src/documents/tests/test_api_uisettings.py

@@ -60,6 +60,10 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
                 },
                 "email_enabled": False,
                 "ai_enabled": False,
+                "remote_ocr": {
+                    "configured": False,
+                    "mode": "always",
+                },
             },
         )
 
@@ -154,6 +158,50 @@ class TestApiUiSettings(DirectoriesMixin, APITestCase):
             str(response.data["settings"]),
         )
 
+    @override_settings(
+        REMOTE_OCR_ENGINE="azureai",
+        REMOTE_OCR_API_KEY="somekey",
+        REMOTE_OCR_ENDPOINT="https://example.cognitiveservices.azure.com",
+        REMOTE_OCR_MODE="workflow_only",
+    )
+    def test_settings_reports_remote_ocr_when_configured(self) -> None:
+        """
+        GIVEN:
+            - A fully configured remote OCR engine in workflow_only mode
+        WHEN:
+            - The ui_settings endpoint is called
+        THEN:
+            - The UI is told remote OCR is available and selective, so it can
+              offer it where it would actually change something
+        """
+        response = self.client.get(self.ENDPOINT, format="json")
+
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        self.assertEqual(
+            response.data["settings"]["remote_ocr"],
+            {"configured": True, "mode": "workflow_only"},
+        )
+
+    @override_settings(
+        REMOTE_OCR_ENGINE="azureai",
+        REMOTE_OCR_API_KEY=None,
+        REMOTE_OCR_ENDPOINT=None,
+    )
+    def test_settings_reports_remote_ocr_incompletely_configured(self) -> None:
+        """
+        GIVEN:
+            - An engine named but missing its endpoint and API key
+        WHEN:
+            - The ui_settings endpoint is called
+        THEN:
+            - It is reported as not configured, matching what the parser
+              registry will actually do
+        """
+        response = self.client.get(self.ENDPOINT, format="json")
+
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        self.assertFalse(response.data["settings"]["remote_ocr"]["configured"])
+
     @override_settings(
         OAUTH_CALLBACK_BASE_URL="http://localhost:8000",
         GMAIL_OAUTH_CLIENT_ID="abc123",

src/documents/tests/test_api_workflows.py

@@ -506,6 +506,280 @@ class TestApiWorkflows(DirectoriesMixin, APITestCase):
 
         self.assertEqual(Workflow.objects.count(), 1)
 
+    def test_api_create_remote_ocr_action_requires_consumption_trigger(
+        self,
+    ) -> None:
+        """
+        GIVEN:
+            - API request to create a workflow with a remote OCR action
+            - No consumption started trigger, so the action could never run
+        WHEN:
+            - API is called
+        THEN:
+            - Correct HTTP 400 response
+            - No objects are created
+        """
+        existing_count = Workflow.objects.count()
+
+        response = self.client.post(
+            self.ENDPOINT,
+            json.dumps(
+                {
+                    "name": "Remote OCR too late",
+                    "order": 1,
+                    "triggers": [
+                        {
+                            "type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
+                        },
+                    ],
+                    "actions": [
+                        {
+                            "type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
+                        },
+                    ],
+                },
+            ),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+        self.assertEqual(Workflow.objects.count(), existing_count)
+
+    def test_api_create_remote_ocr_action_with_consumption_trigger(self) -> None:
+        """
+        GIVEN:
+            - API request to create a workflow with a remote OCR action
+            - A consumption started trigger alongside another trigger type
+        WHEN:
+            - API is called
+        THEN:
+            - The workflow is created, the action applies to consumption only
+        """
+        response = self.client.post(
+            self.ENDPOINT,
+            json.dumps(
+                {
+                    "name": "Remote OCR on consume",
+                    "order": 1,
+                    "triggers": [
+                        {
+                            "type": WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+                            "filter_filename": "*.pdf",
+                        },
+                        {
+                            "type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
+                        },
+                    ],
+                    "actions": [
+                        {
+                            "type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
+                        },
+                    ],
+                },
+            ),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
+
+    def test_api_partial_update_adds_remote_ocr_action(self) -> None:
+        """
+        GIVEN:
+            - An existing workflow with a consumption started trigger
+        WHEN:
+            - A partial update adds a remote OCR action without resubmitting triggers
+        THEN:
+            - The existing trigger is considered and the update succeeds
+        """
+        response = self.client.patch(
+            f"{self.ENDPOINT}{self.workflow.id}/",
+            json.dumps(
+                {
+                    "actions": [
+                        {
+                            "type": WorkflowAction.WorkflowActionType.REMOTE_OCR,
+                        },
+                    ],
+                },
+            ),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        self.assertEqual(
+            self.workflow.actions.get().type,
+            WorkflowAction.WorkflowActionType.REMOTE_OCR,
+        )
+
+    def test_api_partial_update_cannot_remove_remote_ocr_trigger(self) -> None:
+        """
+        GIVEN:
+            - An existing workflow with a remote OCR action
+            - An existing consumption started trigger
+        WHEN:
+            - A partial update replaces the trigger without resubmitting actions
+        THEN:
+            - The existing action is considered and the update is rejected
+        """
+        self.action.type = WorkflowAction.WorkflowActionType.REMOTE_OCR
+        self.action.save()
+
+        response = self.client.patch(
+            f"{self.ENDPOINT}{self.workflow.id}/",
+            json.dumps(
+                {
+                    "triggers": [
+                        {
+                            "type": WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
+                        },
+                    ],
+                },
+            ),
+            content_type="application/json",
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+        self.assertEqual(self.workflow.triggers.get(), self.trigger)
+
+    def _post_ai_suggestions_workflow(self, *, trigger_types, action: dict):
+        def trigger(trigger_type):
+            # consumption triggers require a filter of their own
+            if trigger_type == WorkflowTrigger.WorkflowTriggerType.CONSUMPTION:
+                return {"type": trigger_type, "filter_filename": "*.pdf"}
+            return {"type": trigger_type}
+
+        return self.client.post(
+            self.ENDPOINT,
+            json.dumps(
+                {
+                    "name": "Apply AI suggestions",
+                    "order": 1,
+                    "triggers": [trigger(t) for t in trigger_types],
+                    "actions": [
+                        {
+                            "type": WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
+                            **action,
+                        },
+                    ],
+                },
+            ),
+            content_type="application/json",
+        )
+
+    def test_api_create_apply_ai_suggestions_action(self) -> None:
+        """
+        GIVEN:
+            - API request to create a workflow with an apply AI suggestions
+              action and a valid set of fields
+        WHEN:
+            - API is called
+        THEN:
+            - The workflow is created with the chosen options
+        """
+        response = self._post_ai_suggestions_workflow(
+            trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
+            action={
+                "ai_suggestion_fields": ["title", "tags", "correspondent"],
+                "ai_create_missing": True,
+                "ai_overwrite_existing": True,
+            },
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
+        action = Workflow.objects.get(name="Apply AI suggestions").actions.first()
+        self.assertEqual(
+            action.ai_suggestion_fields,
+            ["title", "tags", "correspondent"],
+        )
+        self.assertTrue(action.ai_create_missing)
+        self.assertTrue(action.ai_overwrite_existing)
+
+    def test_api_create_apply_ai_suggestions_action_requires_fields(self) -> None:
+        """
+        GIVEN:
+            - API request to create an apply AI suggestions action with no
+              fields selected, which could never do anything
+        WHEN:
+            - API is called
+        THEN:
+            - Correct HTTP 400 response
+            - No objects are created
+        """
+        existing_count = Workflow.objects.count()
+
+        response = self._post_ai_suggestions_workflow(
+            trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
+            action={"ai_suggestion_fields": []},
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+        self.assertEqual(Workflow.objects.count(), existing_count)
+
+    def test_api_create_apply_ai_suggestions_action_rejects_unknown_field(
+        self,
+    ) -> None:
+        """
+        GIVEN:
+            - API request to create an apply AI suggestions action naming a
+              field that does not exist
+        WHEN:
+            - API is called
+        THEN:
+            - Correct HTTP 400 response
+        """
+        response = self._post_ai_suggestions_workflow(
+            trigger_types=[WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED],
+            action={"ai_suggestion_fields": ["title", "not_a_field"]},
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+
+    def test_api_create_apply_ai_suggestions_action_rejects_consumption_only(
+        self,
+    ) -> None:
+        """
+        GIVEN:
+            - API request to create an apply AI suggestions action whose only
+              trigger is consumption started, so there is no document content
+              to make suggestions from yet
+        WHEN:
+            - API is called
+        THEN:
+            - Correct HTTP 400 response
+            - No objects are created
+        """
+        existing_count = Workflow.objects.count()
+
+        response = self._post_ai_suggestions_workflow(
+            trigger_types=[WorkflowTrigger.WorkflowTriggerType.CONSUMPTION],
+            action={"ai_suggestion_fields": ["title"]},
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_400_BAD_REQUEST)
+        self.assertEqual(Workflow.objects.count(), existing_count)
+
+    def test_api_create_apply_ai_suggestions_action_allows_extra_consumption_trigger(
+        self,
+    ) -> None:
+        """
+        GIVEN:
+            - API request to create an apply AI suggestions action with a
+              consumption trigger alongside a usable one
+        WHEN:
+            - API is called
+        THEN:
+            - The workflow is created, the action applies to the other trigger
+        """
+        response = self._post_ai_suggestions_workflow(
+            trigger_types=[
+                WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+                WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
+            ],
+            action={"ai_suggestion_fields": ["title"]},
+        )
+
+        self.assertEqual(response.status_code, status.HTTP_201_CREATED)
+
     def test_api_create_workflow_trigger_action_empty_fields(self) -> None:
         """
         GIVEN:

src/documents/tests/test_bulk_edit.py

@@ -1782,3 +1782,56 @@ class TestPDFActions(DirectoriesMixin, TestCase):
 
         self.assertIn("wrong password", str(exc.exception))
         self.assertIn("Error removing password from document", cm.output[0])
+
+
+class TestBulkEditReprocess(DirectoriesMixin, TestCase):
+    def setUp(self) -> None:
+        super().setUp()
+
+        self.doc = Document.objects.create(
+            title="test",
+            checksum="A",
+            mime_type="application/pdf",
+        )
+
+    @mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file")
+    def test_reprocess_defaults_to_local(self, mock_task: mock.Mock) -> None:
+        """
+        GIVEN:
+            - A reprocess request that says nothing about remote OCR
+        WHEN:
+            - reprocess is called
+        THEN:
+            - The task is queued without asking for the remote engine
+        """
+        result = bulk_edit.reprocess([self.doc.id])
+
+        self.assertEqual(result, "OK")
+        mock_task.apply_async.assert_called_once()
+        _, kwargs = mock_task.apply_async.call_args
+        self.assertEqual(
+            kwargs["kwargs"],
+            {"document_id": self.doc.id, "remote_ocr": False},
+        )
+
+    @mock.patch("documents.bulk_edit.update_document_content_maybe_archive_file")
+    def test_reprocess_passes_remote_ocr(self, mock_task: mock.Mock) -> None:
+        """
+        GIVEN:
+            - A reprocess request that explicitly asks for remote OCR
+        WHEN:
+            - reprocess is called
+        THEN:
+            - The request is forwarded to the task for every document
+        """
+        other = Document.objects.create(
+            title="test2",
+            checksum="B",
+            mime_type="application/pdf",
+        )
+
+        bulk_edit.reprocess([self.doc.id, other.id], remote_ocr=True)
+
+        self.assertEqual(mock_task.apply_async.call_count, 2)
+        for call in mock_task.apply_async.call_args_list:
+            self.assertTrue(call.kwargs["kwargs"]["remote_ocr"])

src/documents/tests/test_caching.py

@@ -1,6 +1,13 @@
 import pickle
+from concurrent.futures import ThreadPoolExecutor
+from threading import Event
+from threading import Lock
+
+import pytest
 
 from documents.caching import StoredLRUCache
+from documents.caching import retrieve_llm_suggestions
+from paperless_ai.exceptions import LLMTimeoutError
 
 
 def test_lru_cache_entries() -> None:
@@ -43,3 +50,119 @@ def test_stored_lru_cache_key_ttl(mocker) -> None:
     assert key == "test_key"
     assert timeout == 321
     assert pickle.loads(data) == {"x": "X", "y": "Y"}
+
+
+def test_llm_suggestions_are_generated_once_for_concurrent_requests(mocker) -> None:
+    generation_started = Event()
+    finish_generation = Event()
+    waiter_started = Event()
+    call_lock = Lock()
+    calls = 0
+    suggestions = {"title": "Generated once"}
+    document = mocker.Mock(pk=42)
+    user = mocker.Mock()
+
+    def generate(*args) -> dict:
+        nonlocal calls
+        with call_lock:
+            calls += 1
+        generation_started.set()
+        assert finish_generation.wait(timeout=2)
+        return suggestions
+
+    def wait_for_generation(_interval: float) -> None:
+        waiter_started.set()
+        assert finish_generation.wait(timeout=2)
+
+    mock_get_classification = mocker.patch(
+        "paperless_ai.ai_classifier.get_ai_document_classification",
+        side_effect=generate,
+    )
+    mocker.patch("documents.caching.time.sleep", side_effect=wait_for_generation)
+
+    with ThreadPoolExecutor(max_workers=2) as executor:
+        first = executor.submit(
+            retrieve_llm_suggestions,
+            document,
+            user,
+            None,
+            backend="ollama:model",
+            lock_timeout=10,
+        )
+        assert generation_started.wait(timeout=2)
+        second = executor.submit(
+            retrieve_llm_suggestions,
+            document,
+            user,
+            None,
+            backend="ollama:model",
+            lock_timeout=10,
+        )
+        assert waiter_started.wait(timeout=2)
+        finish_generation.set()
+
+        assert first.result(timeout=2) == suggestions
+        assert second.result(timeout=2) == suggestions
+
+    assert calls == 1
+    mock_get_classification.assert_called_once_with(document, user, None)
+
+
+def test_llm_suggestions_waiter_does_not_rerun_a_failed_generation(mocker) -> None:
+    """
+    A request queued behind a generation that fails should give up, not take
+    its turn at re-running a query that just failed.
+    """
+    generation_started = Event()
+    fail_generation = Event()
+    waiter_started = Event()
+    call_lock = Lock()
+    calls = 0
+    document = mocker.Mock(pk=43)
+    user = mocker.Mock()
+
+    def generate(*args) -> dict:
+        nonlocal calls
+        with call_lock:
+            calls += 1
+        generation_started.set()
+        assert fail_generation.wait(timeout=2)
+        raise ValueError("Unknown model")
+
+    def wait_for_generation(_interval: float) -> None:
+        waiter_started.set()
+        assert fail_generation.wait(timeout=2)
+
+    mocker.patch(
+        "paperless_ai.ai_classifier.get_ai_document_classification",
+        side_effect=generate,
+    )
+    mocker.patch("documents.caching.time.sleep", side_effect=wait_for_generation)
+
+    with ThreadPoolExecutor(max_workers=2) as executor:
+        first = executor.submit(
+            retrieve_llm_suggestions,
+            document,
+            user,
+            None,
+            backend="ollama:model",
+            lock_timeout=10,
+        )
+        assert generation_started.wait(timeout=2)
+        second = executor.submit(
+            retrieve_llm_suggestions,
+            document,
+            user,
+            None,
+            backend="ollama:model",
+            lock_timeout=10,
+        )
+        assert waiter_started.wait(timeout=2)
+        fail_generation.set()
+
+        with pytest.raises(ValueError, match="Unknown model"):
+            first.result(timeout=2)
+        with pytest.raises(LLMTimeoutError):
+            second.result(timeout=2)
+
+    assert calls == 1

src/documents/tests/test_consumer.py

@@ -1559,6 +1559,72 @@ class PostConsumeTestCase(DirectoriesMixin, GetConsumerMixin, TestCase):
                         consumer.run_post_consume_script(doc)
 
 
+class TestConsumerRemoteOCR(
+    DirectoriesMixin,
+    FileSystemAssertsMixin,
+    GetConsumerMixin,
+    TestCase,
+):
+    """
+    The consumer resolves the remote OCR mode and the per-document request from
+    workflows into the allow_remote flag it hands to the parser registry.
+    """
+
+    def setUp(self) -> None:
+        super().setUp()
+
+        patcher = mock.patch("documents.consumer.get_parser_registry")
+        self.mock_registry = patcher.start()
+        self.mock_registry.return_value.get_parser_for_file.return_value = DummyParser
+        self.addCleanup(patcher.stop)
+
+    def _consume(self, *, overrides: DocumentMetadataOverrides | None = None) -> bool:
+        src = (
+            Path(__file__).parent
+            / "samples"
+            / "documents"
+            / "originals"
+            / "0000001.pdf"
+        )
+        dst = self.dirs.scratch_dir / "sample.pdf"
+        shutil.copy(src, dst)
+
+        with self.get_consumer(dst, overrides=overrides) as consumer:
+            consumer.run()
+
+        _, kwargs = self.mock_registry.return_value.get_parser_for_file.call_args
+        return kwargs["allow_remote"]
+
+    @override_settings(REMOTE_OCR_MODE="always")
+    def test_always_mode_allows_remote(self) -> None:
+        """
+        GIVEN: Remote OCR mode is 'always'.
+        WHEN:  A document is consumed without any workflow asking for it.
+        THEN:  The registry is allowed to pick the remote parser.
+        """
+        self.assertTrue(self._consume())
+
+    @override_settings(REMOTE_OCR_MODE="workflow_only")
+    def test_workflow_only_mode_denies_remote_by_default(self) -> None:
+        """
+        GIVEN: Remote OCR mode is 'workflow_only'.
+        WHEN:  A document is consumed and nothing asked for remote OCR.
+        THEN:  The remote parser is excluded.
+        """
+        self.assertFalse(self._consume())
+
+    @override_settings(REMOTE_OCR_MODE="workflow_only")
+    def test_workflow_only_mode_allows_remote_when_requested(self) -> None:
+        """
+        GIVEN: Remote OCR mode is 'workflow_only'.
+        WHEN:  A workflow set remote_ocr on the metadata overrides.
+        THEN:  The registry is allowed to pick the remote parser.
+        """
+        self.assertTrue(
+            self._consume(overrides=DocumentMetadataOverrides(remote_ocr=True)),
+        )
+
+
 class TestMetadataOverrides(TestCase):
     def test_update_skip_asn_if_exists(self) -> None:
         base = DocumentMetadataOverrides()
@@ -1566,6 +1632,20 @@ class TestMetadataOverrides(TestCase):
         base.update(incoming)
         self.assertTrue(base.skip_asn_if_exists)
 
+    def test_update_remote_ocr(self) -> None:
+        base = DocumentMetadataOverrides()
+        base.update(DocumentMetadataOverrides(remote_ocr=True))
+        self.assertTrue(base.remote_ocr)
+
+    def test_update_remote_ocr_is_not_unset(self) -> None:
+        """
+        A later workflow that says nothing must not undo an earlier one that
+        asked for remote OCR.
+        """
+        base = DocumentMetadataOverrides(remote_ocr=True)
+        base.update(DocumentMetadataOverrides())
+        self.assertTrue(base.remote_ocr)
+
     def test_update_actor_and_version_label(self) -> None:
         base = DocumentMetadataOverrides(
             actor_id=1,

src/documents/tests/test_task_signals.py

@@ -385,6 +385,25 @@ class TestTaskFailureHandler:
         task_failure_handler(task_id=None, exception=ValueError("x"), traceback=None)
 
 
+@pytest.mark.django_db
+class TestApplyAiSuggestionsTracking:
+    def test_records_the_document_it_is_for(self) -> None:
+        """
+        The action queues one task per document, so the tracked record notes
+        which document it is for -- otherwise a bulk run is an indistinguishable
+        wall of identical entries in the tasks list.
+        """
+        task_id = send_publish(
+            "documents.tasks.apply_ai_suggestions",
+            (),
+            {"action_id": 1, "document_id": 42},
+        )
+
+        task = PaperlessTask.objects.get(task_id=task_id)
+        assert task.task_type == PaperlessTask.TaskType.APPLY_AI_SUGGESTIONS
+        assert task.input_data == {"document_id": 42}
+
+
 @pytest.mark.django_db
 class TestTaskRevokedHandler:
     def test_marks_task_revoked(self, mocker: pytest_mock.MockerFixture) -> None:

src/documents/tests/test_tasks.py

@@ -14,6 +14,7 @@ from documents.models import Correspondent
 from documents.models import Document
 from documents.models import DocumentType
 from documents.models import Tag
+from documents.models import WorkflowAction
 from documents.sanity_checker import SanityCheckFailedException
 from documents.sanity_checker import SanityCheckMessages
 from documents.tests.test_classifier import dummy_preprocess
@@ -287,6 +288,45 @@ class TestUpdateContent(DirectoriesMixin, TestCase):
         self.assertNotEqual(Document.objects.get(pk=doc.pk).content, "test")
 
 
+class TestUpdateContentRemoteOCR(DirectoriesMixin, TestCase):
+    """
+    Consumption workflows do not run on reprocess, so the remote parser is
+    used only in 'always' mode or when the caller explicitly asks for it.
+    """
+
+    def setUp(self) -> None:
+        super().setUp()
+
+        patcher = mock.patch("documents.tasks.get_parser_registry")
+        self.mock_registry = patcher.start()
+        self.mock_registry.return_value.get_parser_for_file.return_value = None
+        self.addCleanup(patcher.stop)
+
+        self.doc = Document.objects.create(
+            title="test",
+            content="my document",
+            checksum="wow",
+            mime_type="application/pdf",
+        )
+
+    def _allow_remote(self, **kwargs) -> bool:
+        tasks.update_document_content_maybe_archive_file(self.doc.pk, **kwargs)
+        _, call_kwargs = self.mock_registry.return_value.get_parser_for_file.call_args
+        return call_kwargs["allow_remote"]
+
+    @override_settings(REMOTE_OCR_MODE="always")
+    def test_always_mode_allows_remote(self) -> None:
+        self.assertTrue(self._allow_remote())
+
+    @override_settings(REMOTE_OCR_MODE="workflow_only")
+    def test_workflow_only_mode_denies_remote_by_default(self) -> None:
+        self.assertFalse(self._allow_remote())
+
+    @override_settings(REMOTE_OCR_MODE="workflow_only")
+    def test_workflow_only_mode_allows_remote_when_requested(self) -> None:
+        self.assertTrue(self._allow_remote(remote_ocr=True))
+
+
 class TestAIIndex(DirectoriesMixin, TestCase):
     @override_settings(
         AI_ENABLED=True,
@@ -408,3 +448,110 @@ class TestAIIndex(DirectoriesMixin, TestCase):
                 rebuild=False,
                 document_ids=doc_ids,
             )
+
+
+class TestApplyAISuggestionsTask(DirectoriesMixin, TestCase):
+    def setUp(self) -> None:
+        super().setUp()
+        self.doc = Document.objects.create(
+            title="doc",
+            content="content",
+            checksum="apply-ai-suggestions",
+        )
+        self.action = WorkflowAction.objects.create(
+            type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
+            ai_suggestion_fields=[WorkflowAction.AISuggestionField.TITLE],
+        )
+
+    def test_reindexes_without_sending_document_updated(self) -> None:
+        """
+        GIVEN:
+            - An apply AI suggestions action that changes the document
+        WHEN:
+            - The task runs
+        THEN:
+            - The search index and caches are refreshed directly, deliberately
+              not via the document_updated signal: that re-runs updated
+              workflows, which for this action means queueing another LLM
+              query for a document it just changed, forever
+        """
+        with (
+            mock.patch(
+                "documents.workflows.ai.apply_ai_suggestions_to_document",
+                return_value=["title"],
+            ),
+            mock.patch("documents.tasks.index_document") as index_document,
+            mock.patch("documents.tasks.clear_document_caches") as clear_caches,
+            mock.patch("documents.tasks.document_updated") as document_updated,
+        ):
+            tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
+
+        index_document.delay.assert_called_once_with(self.doc.pk)
+        clear_caches.assert_called_once_with(self.doc.pk)
+        document_updated.send.assert_not_called()
+
+    def test_no_changes_skips_reindex(self) -> None:
+        """
+        GIVEN:
+            - An apply AI suggestions action that changes nothing
+        WHEN:
+            - The task runs
+        THEN:
+            - No reindexing work is queued
+        """
+        with (
+            mock.patch(
+                "documents.workflows.ai.apply_ai_suggestions_to_document",
+                return_value=[],
+            ),
+            mock.patch("documents.tasks.index_document") as index_document,
+        ):
+            tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
+
+        index_document.delay.assert_not_called()
+
+    @override_settings(AI_ENABLED=True, LLM_EMBEDDING_BACKEND="huggingface")
+    def test_updates_llm_index_when_enabled(self) -> None:
+        """
+        GIVEN:
+            - An apply AI suggestions action that changes the document
+            - The LLM index is enabled
+        WHEN:
+            - The task runs
+        THEN:
+            - The document is updated in the LLM index too
+        """
+        with (
+            mock.patch(
+                "documents.workflows.ai.apply_ai_suggestions_to_document",
+                return_value=["title"],
+            ),
+            mock.patch("documents.tasks.index_document"),
+            mock.patch(
+                "documents.tasks.update_document_in_llm_index",
+            ) as update_in_llm_index,
+        ):
+            tasks.apply_ai_suggestions(self.action.pk, self.doc.pk)
+
+        update_in_llm_index.apply_async.assert_called_once()
+
+    def test_deleted_document_is_a_noop(self) -> None:
+        """
+        GIVEN:
+            - A document that was deleted between the workflow running and the
+              queued task starting
+        WHEN:
+            - The task runs
+        THEN:
+            - It logs and exits rather than raising
+        """
+        with (
+            mock.patch(
+                "documents.workflows.ai.apply_ai_suggestions_to_document",
+            ) as apply_suggestions,
+            self.assertLogs("paperless.tasks", level="WARNING") as cm,
+        ):
+            tasks.apply_ai_suggestions(self.action.pk, self.doc.pk + 1000)
+
+        apply_suggestions.assert_not_called()
+        self.assertIn("no longer exists", "".join(cm.output))

src/documents/tests/test_views.py

@@ -441,7 +441,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
         self.assertEqual(response.json()["tags"], [])
         self.assertEqual(response.json()["suggested_tags"], [])
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -491,7 +491,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             None,
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -529,7 +529,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             "KI Title",
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -568,7 +568,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             "Titre IA",
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -604,7 +604,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             ),
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="openai-like",
@@ -633,7 +633,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="openai-like",
@@ -660,7 +660,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
             get_llm_suggestion_cache(self.document.pk, backend="openai-like"),
         )
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -698,7 +698,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
         self.assertEqual(response.json()["tags"], [self.tag1.pk])
         self.assertEqual(response.json()["suggested_tags"], ["Follow-up"])
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",
@@ -737,7 +737,7 @@ class TestAISuggestions(DirectoriesMixin, TestCase):
         self.assertEqual(response.json()["tags"], [self.tag1.pk])
         self.assertEqual(response.json()["suggested_tags"], [])
 
-    @patch("documents.views.get_ai_document_classification")
+    @patch("paperless_ai.ai_classifier.get_ai_document_classification")
     @override_settings(
         AI_ENABLED=True,
         LLM_BACKEND="mock_backend",

src/documents/tests/test_workflows.py

@@ -31,7 +31,10 @@ from documents.file_handling import create_source_path_directory
 from documents.file_handling import generate_filename
 from documents.file_handling import generate_unique_filename
 from documents.signals.handlers import run_workflows
+from documents.workflows.ai import apply_ai_suggestions_to_document
 from documents.workflows.webhooks import send_webhook
+from paperless_ai.base_model import ClassificationSuggestions
+from paperless_ai.exceptions import LLMTimeoutError
 
 if TYPE_CHECKING:
     from django.db.models import QuerySet
@@ -5409,3 +5412,560 @@ class TestDateWorkflowLocalization(
             document = Document.objects.first()
             assert document is not None
             assert document.title == expected_title
+
+
+class TestRemoteOCRWorkflowAction(DirectoriesMixin, SampleDirMixin, APITestCase):
+    def _make_workflow(self, trigger_type) -> None:
+        trigger = WorkflowTrigger.objects.create(type=trigger_type)
+        action = WorkflowAction.objects.create(
+            type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
+        )
+        w = Workflow.objects.create(name="Remote OCR", order=0)
+        w.triggers.add(trigger)
+        w.actions.add(action)
+        w.save()
+
+    def test_consumption_trigger_requests_remote_ocr(self) -> None:
+        """
+        GIVEN:
+            - A consumption workflow with a remote OCR action
+        WHEN:
+            - A matching document is consumed
+        THEN:
+            - The overrides ask for remote OCR, which is what the consumer
+              reads when choosing a parser
+        """
+        self._make_workflow(WorkflowTrigger.WorkflowTriggerType.CONSUMPTION)
+
+        test_file = shutil.copy(
+            self.SAMPLE_DIR / "simple.pdf",
+            self.dirs.scratch_dir / "simple.pdf",
+        )
+        overrides = DocumentMetadataOverrides()
+
+        run_workflows(
+            WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+            ConsumableDocument(
+                source=DocumentSource.ConsumeFolder,
+                original_file=test_file,
+            ),
+            overrides=overrides,
+        )
+
+        self.assertTrue(overrides.remote_ocr)
+
+    def test_other_trigger_types_are_ignored(self) -> None:
+        """
+        GIVEN:
+            - A workflow with a remote OCR action that also has a
+              non-consumption trigger, which is a valid combination
+        WHEN:
+            - The non-consumption trigger fires
+        THEN:
+            - The action is skipped, since the document has already been
+              parsed by this point
+        """
+        trigger = WorkflowTrigger.objects.create(
+            type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+        )
+        updated_trigger = WorkflowTrigger.objects.create(
+            type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
+        )
+        action = WorkflowAction.objects.create(
+            type=WorkflowAction.WorkflowActionType.REMOTE_OCR,
+        )
+        w = Workflow.objects.create(name="Remote OCR", order=0)
+        w.triggers.add(trigger, updated_trigger)
+        w.actions.add(action)
+        w.save()
+
+        doc = Document.objects.create(
+            title="sample test",
+            original_filename="sample.pdf",
+        )
+
+        with self.assertLogs("paperless.handlers", level="DEBUG") as cm:
+            run_workflows(
+                WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
+                doc,
+            )
+
+        self.assertIn("only applies to consumption triggers", "".join(cm.output))
+
+
+SUGGESTIONS: ClassificationSuggestions = {
+    "title": "Suggested Title",
+    "tags": {
+        "existing_ids": [],
+        "new_names": ["Existing Tag", "Suggested Tag"],
+    },
+    "correspondents": {
+        "existing_ids": [],
+        "new_names": ["Existing Correspondent", "Suggested Correspondent"],
+    },
+    "document_types": {
+        "existing_ids": [],
+        "new_names": ["Suggested Document Type"],
+    },
+    "storage_paths": {
+        "existing_ids": [],
+        "new_names": ["Suggested Storage Path"],
+    },
+    "dates": ["2024-03-05"],
+}
+
+ALL_SUGGESTION_FIELDS = [
+    WorkflowAction.AISuggestionField.TITLE,
+    WorkflowAction.AISuggestionField.TAGS,
+    WorkflowAction.AISuggestionField.CORRESPONDENT,
+    WorkflowAction.AISuggestionField.DOCUMENT_TYPE,
+    WorkflowAction.AISuggestionField.STORAGE_PATH,
+    WorkflowAction.AISuggestionField.CREATED,
+]
+
+
+@override_settings(AI_ENABLED=True)
+class TestApplyAISuggestionsWorkflowAction(
+    DirectoriesMixin,
+    SampleDirMixin,
+    APITestCase,
+):
+    def setUp(self) -> None:
+        super().setUp()
+        self.user = User.objects.create(username="ai-user")
+        self.doc = Document.objects.create(
+            title="original.pdf",
+            content="the document content",
+            checksum="ai-suggestions-checksum",
+            mime_type="application/pdf",
+            created=datetime.date(2020, 1, 1),
+            owner=self.user,
+        )
+
+    def make_action(self, **kwargs) -> WorkflowAction:
+        return WorkflowAction.objects.create(
+            type=WorkflowAction.WorkflowActionType.APPLY_AI_SUGGESTIONS,
+            ai_suggestion_fields=kwargs.pop(
+                "ai_suggestion_fields",
+                ALL_SUGGESTION_FIELDS,
+            ),
+            **kwargs,
+        )
+
+    def make_workflow(self, action: WorkflowAction, trigger_type) -> Workflow:
+        trigger = WorkflowTrigger.objects.create(type=trigger_type)
+        w = Workflow.objects.create(name="Apply AI suggestions", order=0)
+        w.triggers.add(trigger)
+        w.actions.add(action)
+        w.save()
+        return w
+
+    def apply(
+        self,
+        action: WorkflowAction,
+        suggestions: ClassificationSuggestions = SUGGESTIONS,
+    ) -> list[str]:
+        with mock.patch(
+            "documents.workflows.ai.get_ai_document_classification",
+            return_value=suggestions,
+        ):
+            changed = apply_ai_suggestions_to_document(action, self.doc)
+        self.doc.refresh_from_db()
+        return changed
+
+    def test_document_added_trigger_queues_task(self) -> None:
+        """
+        GIVEN:
+            - A document added workflow with an apply AI suggestions action
+        WHEN:
+            - A matching document is added
+        THEN:
+            - The work is queued rather than run inline, so a slow LLM query
+              cannot stall the rest of the workflow run
+        """
+        action = self.make_action()
+        self.make_workflow(action, WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED)
+
+        with mock.patch("documents.tasks.apply_ai_suggestions.delay") as delay:
+            run_workflows(
+                WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
+                self.doc,
+            )
+
+        delay.assert_called_once_with(action_id=action.pk, document_id=self.doc.pk)
+
+    def test_consumption_trigger_is_ignored(self) -> None:
+        """
+        GIVEN:
+            - A workflow with an apply AI suggestions action and a consumption
+              trigger alongside a valid one
+        WHEN:
+            - The consumption trigger fires
+        THEN:
+            - The action is skipped, since the document has not been parsed
+              yet and so has no content to make suggestions from
+        """
+        action = self.make_action()
+        w = self.make_workflow(
+            action,
+            WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
+        )
+        w.triggers.add(
+            WorkflowTrigger.objects.create(
+                type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+            ),
+        )
+
+        test_file = shutil.copy(
+            self.SAMPLE_DIR / "simple.pdf",
+            self.dirs.scratch_dir / "simple.pdf",
+        )
+
+        with (
+            mock.patch("documents.tasks.apply_ai_suggestions.delay") as delay,
+            self.assertLogs("paperless.handlers", level="DEBUG") as cm,
+        ):
+            run_workflows(
+                WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
+                ConsumableDocument(
+                    source=DocumentSource.ConsumeFolder,
+                    original_file=test_file,
+                ),
+                overrides=DocumentMetadataOverrides(),
+            )
+
+        delay.assert_not_called()
+        self.assertIn("does not apply to consumption triggers", "".join(cm.output))
+
+    def test_no_selected_fields_does_nothing(self) -> None:
+        """
+        GIVEN:
+            - An action with no suggestion fields selected
+        WHEN:
+            - The action is applied
+        THEN:
+            - Nothing is changed and it is logged
+        """
+        action = self.make_action(ai_suggestion_fields=[])
+
+        with self.assertLogs("paperless.workflows.ai", level="WARNING") as cm:
+            changed = self.apply(action)
+
+        self.assertEqual(changed, [])
+        self.assertIn("no AI suggestion fields selected", "".join(cm.output))
+
+    @override_settings(AI_ENABLED=False)
+    def test_ai_disabled_does_nothing(self) -> None:
+        """
+        GIVEN:
+            - An action on an install where AI has since been disabled
+        WHEN:
+            - The action is applied
+        THEN:
+            - Nothing is changed and it is logged
+        """
+        action = self.make_action()
+
+        with self.assertLogs("paperless.workflows.ai", level="ERROR") as cm:
+            changed = self.apply(action)
+
+        self.assertEqual(changed, [])
+        self.assertIn("AI is not enabled", "".join(cm.output))
+
+    def test_invalid_configuration_leaves_document_untouched(self) -> None:
+        """
+        GIVEN:
+            - An AI backend that is misconfigured
+        WHEN:
+            - The action is applied
+        THEN:
+            - The failure is logged and the document is left alone. It is not
+              re-raised, because retrying will not fix a bad configuration
+        """
+        action = self.make_action()
+
+        with (
+            mock.patch(
+                "documents.workflows.ai.get_ai_document_classification",
+                side_effect=ValueError("nope"),
+            ),
+            self.assertLogs("paperless.workflows.ai", level="ERROR") as cm,
+        ):
+            changed = apply_ai_suggestions_to_document(action, self.doc)
+
+        self.assertEqual(changed, [])
+        self.doc.refresh_from_db()
+        self.assertEqual(self.doc.title, "original.pdf")
+        self.assertIn("Invalid AI configuration", "".join(cm.output))
+
+    def test_transient_llm_failure_is_raised_for_retry(self) -> None:
+        """
+        GIVEN:
+            - An LLM backend that times out, or rate limits the request
+        WHEN:
+            - The action is applied
+        THEN:
+            - The error propagates so the queued task can back off and retry,
+              rather than silently dropping this document's suggestions
+        """
+        action = self.make_action()
+
+        with (
+            mock.patch(
+                "documents.workflows.ai.get_ai_document_classification",
+                side_effect=LLMTimeoutError(),
+            ),
+            self.assertRaises(LLMTimeoutError),
+        ):
+            apply_ai_suggestions_to_document(action, self.doc)
+
+        self.doc.refresh_from_db()
+        self.assertEqual(self.doc.title, "original.pdf")
+
+    def test_only_matching_objects_are_applied(self) -> None:
+        """
+        GIVEN:
+            - An action without create missing, and only some of the suggested
+              objects existing
+        WHEN:
+            - The action is applied
+        THEN:
+            - Only the existing objects are assigned, unmatched suggestions are
+              dropped rather than creating anything
+        """
+        tag = Tag.objects.create(name="Existing Tag", owner=self.user)
+        correspondent = Correspondent.objects.create(
+            name="Existing Correspondent",
+            owner=self.user,
+        )
+        action = self.make_action(ai_overwrite_existing=True)
+
+        changed = self.apply(action)
+
+        self.assertEqual(self.doc.correspondent, correspondent)
+        self.assertEqual(list(self.doc.tags.all()), [tag])
+        # Nothing matched for these and create missing is off
+        self.assertIsNone(self.doc.document_type)
+        self.assertIsNone(self.doc.storage_path)
+        self.assertNotIn("document_type", changed)
+        self.assertEqual(Tag.objects.count(), 1)
+        self.assertEqual(Correspondent.objects.count(), 1)
+
+    def test_existing_id_suggestions_are_applied(self) -> None:
+        """
+        GIVEN:
+            - AI suggestions that select existing taxonomy candidates by ID
+        WHEN:
+            - The suggestions are applied
+        THEN:
+            - Each selected object is assigned to the document
+        """
+        tag = Tag.objects.create(name="Existing Tag", owner=self.user)
+        correspondent = Correspondent.objects.create(
+            name="Existing Correspondent",
+            owner=self.user,
+        )
+        document_type = DocumentType.objects.create(
+            name="Existing Document Type",
+            owner=self.user,
+        )
+        storage_path = StoragePath.objects.create(
+            name="Existing Storage Path",
+            path="{{ title }}",
+            owner=self.user,
+        )
+        action = self.make_action(ai_overwrite_existing=True)
+        suggestions: ClassificationSuggestions = {
+            **SUGGESTIONS,
+            "tags": {"existing_ids": [tag.pk], "new_names": []},
+            "correspondents": {
+                "existing_ids": [correspondent.pk],
+                "new_names": [],
+            },
+            "document_types": {
+                "existing_ids": [document_type.pk],
+                "new_names": [],
+            },
+            "storage_paths": {
+                "existing_ids": [storage_path.pk],
+                "new_names": [],
+            },
+        }
+
+        changed = self.apply(action, suggestions)
+
+        self.assertEqual(list(self.doc.tags.all()), [tag])
+        self.assertEqual(self.doc.correspondent, correspondent)
+        self.assertEqual(self.doc.document_type, document_type)
+        self.assertEqual(self.doc.storage_path, storage_path)
+        self.assertTrue(
+            {"tags", "correspondent", "document_type", "storage_path"} <= set(changed),
+        )
+
+    def test_create_missing_creates_objects_owned_by_document_owner(self) -> None:
+        """
+        GIVEN:
+            - An action with create missing enabled
+        WHEN:
+            - The action is applied and suggestions match nothing
+        THEN:
+            - Tags, correspondents and document types are created, owned by the
+              document owner so they stay private to them
+            - Storage paths are never created, since a path template cannot be
+              inferred from a name
+        """
+        action = self.make_action(
+            ai_create_missing=True,
+            ai_overwrite_existing=True,
+        )
+
+        changed = self.apply(action)
+
+        self.assertEqual(
+            sorted(t.name for t in self.doc.tags.all()),
+            ["Existing Tag", "Suggested Tag"],
+        )
+        self.assertEqual(self.doc.correspondent.name, "Existing Correspondent")
+        self.assertEqual(self.doc.correspondent.owner, self.user)
+        self.assertEqual(self.doc.document_type.name, "Suggested Document Type")
+        self.assertEqual(self.doc.document_type.owner, self.user)
+
+        self.assertIsNone(self.doc.storage_path)
+        self.assertFalse(StoragePath.objects.exists())
+        self.assertNotIn("storage_path", changed)
+
+    def test_overwrite_disabled_keeps_existing_values(self) -> None:
+        """
+        GIVEN:
+            - An action without overwrite existing
+            - A document that already has a title, created date and
+              correspondent
+        WHEN:
+            - The action is applied
+        THEN:
+            - The existing values are kept, only the empty document type is
+              filled in
+        """
+        existing = Correspondent.objects.create(name="Mine", owner=self.user)
+        self.doc.correspondent = existing
+        self.doc.save()
+        action = self.make_action(ai_create_missing=True)
+
+        changed = self.apply(action)
+
+        self.assertEqual(self.doc.title, "original.pdf")
+        self.assertEqual(self.doc.created, datetime.date(2020, 1, 1))
+        self.assertEqual(self.doc.correspondent, existing)
+        self.assertEqual(self.doc.document_type.name, "Suggested Document Type")
+        self.assertNotIn("title", changed)
+        self.assertNotIn("correspondent", changed)
+
+    def test_overwrite_enabled_replaces_existing_values(self) -> None:
+        """
+        GIVEN:
+            - An action with overwrite existing
+            - A document that already has a title and created date
+        WHEN:
+            - The action is applied
+        THEN:
+            - The suggested values replace them
+        """
+        action = self.make_action(
+            ai_create_missing=True,
+            ai_overwrite_existing=True,
+        )
+
+        changed = self.apply(action)
+
+        self.assertEqual(self.doc.title, "Suggested Title")
+        self.assertEqual(self.doc.created, datetime.date(2024, 3, 5))
+        self.assertIn("title", changed)
+        self.assertIn("created", changed)
+
+    def test_tags_are_added_not_replaced(self) -> None:
+        """
+        GIVEN:
+            - A document that already has a tag unrelated to the suggestions
+        WHEN:
+            - The action is applied with overwrite existing enabled
+        THEN:
+            - The existing tag is kept, since suggested tags are always
+              additive regardless of the overwrite setting
+        """
+        kept = Tag.objects.create(name="Do Not Remove", owner=self.user)
+        self.doc.tags.add(kept)
+        Tag.objects.create(name="Existing Tag", owner=self.user)
+        action = self.make_action(ai_overwrite_existing=True)
+
+        self.apply(action)
+
+        self.assertEqual(
+            sorted(t.name for t in self.doc.tags.all()),
+            ["Do Not Remove", "Existing Tag"],
+        )
+
+    def test_unselected_fields_are_untouched(self) -> None:
+        """
+        GIVEN:
+            - An action that only selects the title
+        WHEN:
+            - The action is applied
+        THEN:
+            - Only the title changes, even though the LLM suggested everything
+        """
+        action = self.make_action(
+            ai_suggestion_fields=[WorkflowAction.AISuggestionField.TITLE],
+            ai_create_missing=True,
+            ai_overwrite_existing=True,
+        )
+
+        changed = self.apply(action)
+
+        self.assertEqual(changed, ["title"])
+        self.assertEqual(self.doc.title, "Suggested Title")
+        self.assertEqual(self.doc.tags.count(), 0)
+        self.assertIsNone(self.doc.correspondent)
+        self.assertEqual(self.doc.created, datetime.date(2020, 1, 1))
+
+    def test_another_users_private_objects_are_not_matched(self) -> None:
+        """
+        GIVEN:
+            - A suggested tag name that exists, but is owned by someone else
+        WHEN:
+            - The action is applied
+        THEN:
+            - It is not assigned, because the document owner cannot see it
+        """
+        other = User.objects.create(username="someone-else")
+        Tag.objects.create(name="Existing Tag", owner=other)
+        action = self.make_action(
+            ai_suggestion_fields=[WorkflowAction.AISuggestionField.TAGS],
+        )
+
+        self.apply(action)
+
+        self.assertEqual(self.doc.tags.count(), 0)
+
+    def test_unparsable_dates_are_skipped(self) -> None:
+        """
+        GIVEN:
+            - Suggested dates that are not all valid
+        WHEN:
+            - The action is applied
+        THEN:
+            - The first usable date is applied and the rest ignored
+        """
+        action = self.make_action(
+            ai_suggestion_fields=[WorkflowAction.AISuggestionField.CREATED],
+            ai_overwrite_existing=True,
+        )
+
+        with mock.patch(
+            "documents.workflows.ai.get_ai_document_classification",
+            return_value={**SUGGESTIONS, "dates": ["not a date", "2019-07-04"]},
+        ):
+            changed = apply_ai_suggestions_to_document(action, self.doc)
+
+        self.doc.refresh_from_db()
+        self.assertEqual(changed, ["created"])
+        self.assertEqual(self.doc.created, datetime.date(2019, 7, 4))

src/documents/views.py

@@ -16,7 +16,6 @@ from time import mktime
 from time import sleep
 from typing import TYPE_CHECKING
 from typing import Any
-from typing import Final
 from typing import Literal
 from typing import NamedTuple
 from unicodedata import normalize
@@ -116,7 +115,7 @@ from documents.caching import get_metadata_cache
 from documents.caching import get_suggestion_cache
 from documents.caching import refresh_metadata_cache
 from documents.caching import refresh_suggestions_cache
-from documents.caching import set_llm_suggestions_cache
+from documents.caching import retrieve_llm_suggestions
 from documents.caching import set_metadata_cache
 from documents.caching import set_suggestions_cache
 from documents.classifier import load_classifier
@@ -240,12 +239,14 @@ from paperless import version
 from paperless.celery import app as celery_app
 from paperless.config import AIConfig
 from paperless.config import GeneralConfig
+from paperless.config import RemoteOCRConfig
 from paperless.models import ApplicationConfiguration
 from paperless.parsers.registry import get_parser_registry
+from paperless.parsers.remote import RemoteEngineConfig
 from paperless.serialisers import GroupSerializer
 from paperless.serialisers import UserSerializer
 from paperless.views import StandardPagination
-from paperless_ai.ai_classifier import get_ai_document_classification
+from paperless_ai.ai_classifier import get_llm_output_language
 from paperless_ai.chat import stream_chat_with_documents
 from paperless_ai.exceptions import LLMTimeoutError
 from paperless_ai.matching import extract_unmatched_names
@@ -280,40 +281,17 @@ logger = logging.getLogger("paperless.api")
 _TANTIVY_INTERSECT_THRESHOLD = 5_000
 _TANTIVY_SEARCH_PARAM_NAMES = ("text", "title_search", "query", "more_like_id")
 
-# whoosh-compat's fieldname tagger (used only for SearchMode.QUERY, via the
-# whoosh grammar in parse_user_query) is O(n^2) in plain word characters:
-# measured at ~0.96s/10k chars, ~3.67s/20k, ~14.4s/40k against the real field
-# registry. Django's DATA_UPLOAD_MAX_MEMORY_SIZE default (2.5 MB) does not
-# bound this on the POST-body selection-filter path, so an unbounded query
-# is a single-request CPU exhaustion vector. 4096 chars caps the worst case
-# at roughly 0.16s (quadratic extrapolation from the measurements above),
-# far beyond any plausible hand-typed advanced query, while still being fast
-# enough to absorb inside a request handler. Applied to all three modes at
-# this shared choke point: TEXT and TITLE route through simple_search_tokens
-# instead and measure linear even at 20k chars, so the cap is hygiene for
-# them, not a fix, but a single limit here is simpler than one exemption.
-# Not exposed as a PAPERLESS_* setting: this is a hard security boundary,
-# not a tunable, and a raisable ceiling would let a misconfiguration
-# reintroduce the exact hazard this exists to close.
-_MAX_QUERY_LENGTH: Final[int] = 4096
-
 
 def _get_tantivy_query_and_mode(params):
-    from documents.search import QueryTooLongError
     from documents.search import SearchMode
 
     if "text" in params:
-        raw, mode = str(params["text"]), SearchMode.TEXT
-    elif "title_search" in params:
-        raw, mode = str(params["title_search"]), SearchMode.TITLE
-    elif "query" in params:
-        raw, mode = str(params["query"]), SearchMode.QUERY
-    else:
-        return None  # pragma: no cover
-
-    if len(raw) > _MAX_QUERY_LENGTH:
-        raise QueryTooLongError(len(raw), _MAX_QUERY_LENGTH)
-    return raw, mode
+        return str(params["text"]), SearchMode.TEXT
+    if "title_search" in params:
+        return str(params["title_search"]), SearchMode.TITLE
+    if "query" in params:
+        return str(params["query"]), SearchMode.QUERY
+    return None  # pragma: no cover
 
 
 def _get_more_like_id(query_params: dict[str, Any], user: User | None) -> int:
@@ -687,20 +665,6 @@ class TagViewSet(PermissionsAwareDocumentCountMixin, ModelViewSet[Tag]):
             update_document_parent_tags(tag, new_parent)
 
 
-def _get_llm_output_language(ai_config: AIConfig, request) -> str | None:
-    output_language = ai_config.llm_output_language
-    if (
-        not output_language
-        and hasattr(request.user, "ui_settings")
-        and isinstance(
-            request.user.ui_settings.settings,
-            dict,
-        )
-    ):
-        output_language = request.user.ui_settings.settings.get("language")
-    return output_language
-
-
 @extend_schema_view(**generate_object_with_permissions_schema(DocumentTypeSerializer))
 class DocumentTypeViewSet(
     PermissionsAwareDocumentCountMixin,
@@ -1155,7 +1119,8 @@ class DocumentViewSet(
                     "custom_fields",
                     queryset=CustomFieldInstance.objects.select_related("field"),
                 ),
-                "notes",
+                # NotesSerializer nests the author, this avoids query per note
+                Prefetch("notes", queryset=Note.objects.select_related("user")),
             )
         )
 
@@ -1563,7 +1528,10 @@ class DocumentViewSet(
         if not ai_config.ai_enabled:
             return HttpResponseBadRequest("AI is required for this feature")
 
-        output_language = _get_llm_output_language(ai_config=ai_config, request=request)
+        output_language = get_llm_output_language(
+            ai_config=ai_config,
+            user=request.user,
+        )
         llm_cache_backend = ":".join(
             part
             for part in (
@@ -1591,10 +1559,13 @@ class DocumentViewSet(
             llm_suggestions = cached_llm_suggestions.suggestions
         else:
             try:
-                llm_suggestions = get_ai_document_classification(
-                    doc,
-                    request.user,
-                    output_language,
+                llm_suggestions = retrieve_llm_suggestions(
+                    document=doc,
+                    user=request.user,
+                    output_language=output_language,
+                    backend=llm_cache_backend,
+                    # Classification, localization + 30s
+                    lock_timeout=(2 * ai_config.llm_request_timeout) + 30,
                 )
             except ValueError as exc:
                 logger.exception(
@@ -1619,11 +1590,6 @@ class DocumentViewSet(
                     {"ai": [_("AI backend request timed out.")]},
                     status=status.HTTP_503_SERVICE_UNAVAILABLE,
                 )
-            set_llm_suggestions_cache(
-                doc.pk,
-                llm_suggestions,
-                backend=llm_cache_backend,
-            )
 
         tags_choice: TaxonomyChoiceDict = llm_suggestions["tags"]
         correspondents_choice: TaxonomyChoiceDict = llm_suggestions["correspondents"]
@@ -2352,7 +2318,10 @@ class ChatStreamingView(GenericAPIView[Any]):
                 id__in=permitted_document_ids(request.user),
             )
 
-        output_language = _get_llm_output_language(ai_config=ai_config, request=request)
+        output_language = get_llm_output_language(
+            ai_config=ai_config,
+            user=request.user,
+        )
 
         response = StreamingHttpResponse(
             stream_chat_with_documents(
@@ -2445,7 +2414,6 @@ class UnifiedSearchViewSet(DocumentViewSet):
         from documents.search import TantivyBackend
         from documents.search import TantivyRelevanceList
         from documents.search import get_backend
-        from documents.search import search_query_error_messages
 
         def parse_search_params() -> SearchParams:
             """Extract query string, search mode, and ordering from request."""
@@ -2636,10 +2604,15 @@ class UnifiedSearchViewSet(DocumentViewSet):
         except ValidationError:
             raise
         except SearchQueryError as e:
-            # User-fixable query error(s) (e.g. unparsable dates/numbers):
-            # surface every offending field's message, not just the first,
-            # so the user can fix them all in one round-trip.
-            raise ValidationError({"query": search_query_error_messages(e)}) from e
+            # User-fixable query error (e.g. an unparsable date): surface the
+            # specific message so the user can correct it, rather than a generic
+            # 400 or silently empty results.
+            raise ValidationError({"query": [str(e)]}) from e
+        except Exception as e:
+            logger.warning(f"An error occurred listing search results: {e!s}")
+            return HttpResponseBadRequest(
+                "Error listing search results, check logs for more detail.",
+            )
 
     @action(detail=False, methods=["GET"], name="Get Next ASN")
     def next_asn(self, request, *args, **kwargs):
@@ -2775,34 +2748,23 @@ class DocumentSelectionMixin:
                 },
             )
 
-        from documents.search import SearchQueryError
         from documents.search import get_backend
-        from documents.search import search_query_error_messages
 
         filter_name = search_filters[0]
         backend = get_backend()
         search_user = None if user.is_superuser else user
 
-        try:
-            if filter_name == "more_like_id":
-                more_like_doc_id = _get_more_like_id(filters, user)
+        if filter_name == "more_like_id":
+            more_like_doc_id = _get_more_like_id(filters, user)
 
-                search_ids = backend.more_like_this_ids(
-                    more_like_doc_id,
-                    user=search_user,
-                )
-            else:
-                query_str, search_mode = _get_tantivy_query_and_mode(filters)
-                search_ids = backend.search_ids(
-                    query_str,
-                    user=search_user,
-                    search_mode=search_mode,
-                )
-        except SearchQueryError as e:
-            # Same user-fixable-query mapping as the search list endpoint:
-            # a bad date/number in a bulk selection filter is a 400 naming
-            # the value, never a 500.
-            raise ValidationError({"query": search_query_error_messages(e)}) from e
+            search_ids = backend.more_like_this_ids(more_like_doc_id, user=search_user)
+        else:
+            query_str, search_mode = _get_tantivy_query_and_mode(filters)
+            search_ids = backend.search_ids(
+                query_str,
+                user=search_user,
+                search_mode=search_mode,
+            )
 
         return search_ids
 
@@ -2823,8 +2785,11 @@ class DocumentSelectionMixin:
             for key, value in filters.items()
             if key not in _TANTIVY_SEARCH_PARAM_NAMES
         }
+        # Operations are addressed to roots, a caller that wants
+        # to act on a specific version passes its id explicitly instead
         permitted_documents = Document.objects.filter(
             id__in=permitted_document_ids(user),
+            root_document__isnull=True,
         )
         # orm-filtered docs
         filtered_documents = DocumentFilterSet(
@@ -3639,10 +3604,6 @@ class GlobalSearchView(PassUserMixin):
             return HttpResponseBadRequest("Query required")
         if len(query) < 3:
             return HttpResponseBadRequest("Query must be at least 3 characters")
-        if len(query) > _MAX_QUERY_LENGTH:
-            return HttpResponseBadRequest(
-                f"Query must be at most {_MAX_QUERY_LENGTH} characters",
-            )
 
         db_only = request.query_params.get("db_only", False)
 
@@ -4135,6 +4096,11 @@ class UiSettingsView(GenericAPIView[Any]):
 
         ui_settings["auditlog_enabled"] = settings.AUDIT_LOG_ENABLED
 
+        ui_settings["remote_ocr"] = {
+            "configured": RemoteEngineConfig.from_app_config().engine_is_valid(),
+            "mode": RemoteOCRConfig().remote_ocr_mode,
+        }
+
         if settings.GMAIL_OAUTH_ENABLED or settings.OUTLOOK_OAUTH_ENABLED:
             manager = PaperlessMailOAuth2Manager()
             if settings.GMAIL_OAUTH_ENABLED:

src/documents/workflows/ai.py

@@ -0,0 +1,259 @@
+import logging
+from datetime import date
+from datetime import datetime
+from typing import TypeVar
+
+from django.contrib.auth.models import User
+
+from documents.models import Correspondent
+from documents.models import Document
+from documents.models import DocumentType
+from documents.models import MatchingModel
+from documents.models import StoragePath
+from documents.models import Tag
+from documents.models import WorkflowAction
+from paperless.config import AIConfig
+from paperless_ai.ai_classifier import get_ai_document_classification
+from paperless_ai.ai_classifier import get_llm_output_language
+from paperless_ai.matching import extract_unmatched_names
+from paperless_ai.matching import match_correspondents_by_name
+from paperless_ai.matching import match_document_types_by_name
+from paperless_ai.matching import match_storage_paths_by_name
+from paperless_ai.matching import match_tags_by_name
+from paperless_ai.matching import resolve_correspondent_ids
+from paperless_ai.matching import resolve_document_type_ids
+from paperless_ai.matching import resolve_storage_path_ids
+from paperless_ai.matching import resolve_tag_ids
+
+logger = logging.getLogger("paperless.workflows.ai")
+
+AISuggestionField = WorkflowAction.AISuggestionField
+ObjT = TypeVar("ObjT", bound=MatchingModel)
+
+# Tags use m2m relation instead
+DIRECT_FIELDS: dict[str, str] = {
+    AISuggestionField.TITLE: "title",
+    AISuggestionField.CORRESPONDENT: "correspondent",
+    AISuggestionField.DOCUMENT_TYPE: "document_type",
+    AISuggestionField.STORAGE_PATH: "storage_path",
+    AISuggestionField.CREATED: "created",
+}
+
+
+def resolve_date(dates: list[str]) -> date | None:
+    """
+    First usable date out of the suggestions, which are expected as
+    YYYY-MM-DD. Document.created is a DateField, so only one can be applied.
+    """
+    for value in dates:
+        try:
+            return datetime.strptime(value, "%Y-%m-%d").date()
+        except (TypeError, ValueError):
+            logger.debug("Ignoring unparsable suggested date %s", value)
+    return None
+
+
+def resolve_object(
+    model: type[ObjT],
+    names: list[str],
+    matched: list[ObjT],
+    *,
+    create_missing: bool,
+    owner: User | None,
+) -> ObjT | None:
+    """
+    Single object from a suggestion list. The best match if there was one, else
+    optionally a newly-created object. StoragePaths are excluded.
+    """
+    if matched:
+        return matched[0]
+
+    if not create_missing or model is StoragePath:
+        return None
+
+    unmatched = extract_unmatched_names(names, matched)
+    if not unmatched:
+        return None
+
+    # (name, owner) is what MatchingModel is unique on
+    obj, created = model.objects.get_or_create(
+        name=unmatched[0][:128],
+        owner=owner,
+    )
+    if created:
+        logger.info("Created %s '%s' from AI suggestion", model.__name__, obj.name)
+    return obj
+
+
+def resolve_tags(
+    names: list[str],
+    matched: list[Tag],
+    *,
+    create_missing: bool,
+    owner: User | None,
+) -> list[Tag]:
+    """
+    Matched tags, plus newly created ones if create_missing is set.
+    """
+    tags = list(matched)
+    if not create_missing:
+        return tags
+
+    for name in extract_unmatched_names(names, matched):
+        tag, created = Tag.objects.get_or_create(
+            name=name[:128],
+            owner=owner,
+        )
+        if created:
+            logger.info("Created tag '%s' from AI suggestion", tag.name)
+        tags.append(tag)
+    return tags
+
+
+def apply_ai_suggestions_to_document(
+    action: WorkflowAction,
+    document: Document,
+    logging_group=None,
+) -> list[str]:
+    """
+    Get suggestions about `document` and write the chosen fields.
+
+    Returns the names of the fields that were actually changed.
+    """
+    selected = set(action.ai_suggestion_fields or [])
+    if not selected:
+        logger.warning(
+            "Workflow action %s has no AI suggestion fields selected, skipping",
+            action.pk,
+            extra={"group": logging_group},
+        )
+        return []
+
+    ai_config = AIConfig()
+    if not ai_config.ai_enabled:
+        logger.error(
+            "AI is not enabled, cannot apply AI suggestions for document %s",
+            document.pk,
+            extra={"group": logging_group},
+        )
+        return []
+
+    # Workflows run without a user, so we use the document owner
+    owner = document.owner
+
+    try:
+        suggestions = get_ai_document_classification(
+            document,
+            owner,
+            get_llm_output_language(ai_config, owner),
+        )
+    except ValueError:
+        # A bad AI config will not fix itself, so swallow it rather than
+        # letting the caller retry. Timeouts, rate limits, network errors etc
+        # propagate so the queued task can back off and try again.
+        logger.exception(
+            "Invalid AI configuration, cannot get suggestions for document %s",
+            document.pk,
+            extra={"group": logging_group},
+        )
+        return []
+
+    overwrite = action.ai_overwrite_existing
+    create_missing = action.ai_create_missing
+    updated_fields: list[str] = []
+
+    def should_set(field: str) -> bool:
+        # The field is selected and (overwrite or it's empty)
+        return field in selected and (
+            overwrite or getattr(document, DIRECT_FIELDS[field]) in (None, "")
+        )
+
+    if should_set(AISuggestionField.TITLE):
+        title = suggestions["title"].strip()
+        if title:
+            # title is capped at 128 characters
+            document.title = title[:128]
+            updated_fields.append("title")
+
+    if should_set(AISuggestionField.CORRESPONDENT):
+        choice = suggestions["correspondents"]
+        names = choice["new_names"]
+        correspondent = resolve_object(
+            Correspondent,
+            names,
+            resolve_correspondent_ids(choice["existing_ids"], owner)
+            + match_correspondents_by_name(names, owner),
+            create_missing=create_missing,
+            owner=owner,
+        )
+        if correspondent:
+            document.correspondent = correspondent
+            updated_fields.append("correspondent")
+
+    if should_set(AISuggestionField.DOCUMENT_TYPE):
+        choice = suggestions["document_types"]
+        names = choice["new_names"]
+        document_type = resolve_object(
+            DocumentType,
+            names,
+            resolve_document_type_ids(choice["existing_ids"], owner)
+            + match_document_types_by_name(names, owner),
+            create_missing=create_missing,
+            owner=owner,
+        )
+        if document_type:
+            document.document_type = document_type
+            updated_fields.append("document_type")
+
+    if should_set(AISuggestionField.STORAGE_PATH):
+        choice = suggestions["storage_paths"]
+        names = choice["new_names"]
+        storage_path = resolve_object(
+            StoragePath,
+            names,
+            resolve_storage_path_ids(choice["existing_ids"], owner)
+            + match_storage_paths_by_name(names, owner),
+            create_missing=create_missing,
+            owner=owner,
+        )
+        if storage_path:
+            document.storage_path = storage_path
+            updated_fields.append("storage_path")
+
+    if should_set(AISuggestionField.CREATED):
+        created = resolve_date(suggestions["dates"])
+        if created:
+            document.created = created
+            updated_fields.append("created")
+
+    if AISuggestionField.TAGS in selected:
+        choice = suggestions["tags"]
+        names = choice["new_names"]
+        tags = resolve_tags(
+            names,
+            resolve_tag_ids(choice["existing_ids"], owner)
+            + match_tags_by_name(names, owner),
+            create_missing=create_missing,
+            owner=owner,
+        )
+        if tags:
+            # Suggested tags are always added, so overwrite_existing
+            # does not really apply here
+            document.add_nested_tags(tags)
+            updated_fields.append("tags")
+
+    if updated_fields:
+        # save fields and update modified (excluding m2m tags from update_fields)
+        direct_updated_fields = [
+            field for field in updated_fields if field in DIRECT_FIELDS.values()
+        ]
+        document.save(update_fields=[*direct_updated_fields, "modified"])
+
+    logger.info(
+        "Applied AI suggestions %s to document %s",
+        updated_fields or "(none)",
+        document.pk,
+        extra={"group": logging_group},
+    )
+
+    return updated_fields