Create the shared field-definition table consumed by the schema builder
(_schema.py) and the whoosh-compat field registry (_registry.py). This
eliminates drift between what the index exposes and what queries can address.
- Create PublicField frozen dataclass with field metadata
- Define PUBLIC_FIELDS tuple with 16 searchable fields
- Add comprehensive test suite covering field properties
The whoosh-compat pyproject.toml dependency addition is added in a
follow-up commit, with the correct [tantivy] extra, source comment, and a
matching uv.lock update.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The library changed after these were written and more changes are already
decided upstream. Records what is wrong today, what to write toward, the
one question still open, and the fast-JSON-field trap, rather than
silently leaving code that would fail on contact.
Field references became a typed value rather than a dotted string, so
diagnostics carry one too and the registry exposes a single resolver.
Also records that the JSON fields must stay non-fast while existence
checks against a fast JSON field return inverted results.
Diagnostics now carry field and raw_value, so the transition guidance
points at those instead of parsing human-readable message text.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
16 bite-sized, TDD tasks across the design spec's 4-PR stack, each with
a suggested subagent type/model for delegated execution. Test/fixture
code in the acceptance-corpus and API-expansion tasks was verified
against the real codebase (documents/tests/search/conftest.py's
existing backend/index fixtures, test_backend.py's pytestmark
convention, CustomFieldInstance's typed value_text field) rather than
guessed, and the date-grammar parity test's AST-shape assumption was
confirmed by actually running whoosh_compat.parse() against a real
DATE FieldRegistry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Agent review (source-verified against both repos) confirmed the spec's
claims accurate throughout, with one real gap: the JSON-subpath
tantivy-py carve-out (index.parse_query fallback for notes.*/
custom_fields.* until tantivy-py#716 ships) interacts with paperless's
pinned tantivy~=0.26.0 and wasn't mentioned. Also added two footnotes:
FieldRegistry forces date_only=True on any DATE spec regardless of the
PublicField default, and the date-grammar parity audit implicitly
grants new keyword vocabulary as a side effect.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Design for replacing _translate.py/_dates.py with whoosh-compat: shared
field-definition table driving both the Tantivy schema and the query
FieldRegistry, diagnostics->exception mapping (aggregating all errors,
not just the first), a 4-PR stack with no rollout flag, and a
result-level acceptance corpus + date-grammar parity audit as the
safety net instead.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Encodes the settled integration decisions for replacing the
hand-maintained search translation layer with whoosh-compat:
user-typed query surface policy, analyzer seam, diagnostics-before-emit
contract, mandatory date parity audit, test churn, and rollout plan.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
AI Suggestions previously invented near-duplicate metadata because the classification
prompt had no knowledge of the installation's own taxonomy. This surfaces
a small, ranked, permission-filtered set of existing tags/document
types/correspondents/storage paths - drawn from the document's RAG
neighbors plus its own already-assigned metadata - so the model prefers
reusing what already exists.
The LLM response schema now returns existing_ids (IDs of reused
candidates) separately from new_names (genuinely new suggestions).
Only new_names goes through localization and fuzzy name-matching;
existing_ids is resolved deterministically and never touched by the
localization pass, so exact matches can no longer be silently
corrupted by translation.
* Feature: Allow configuring the compression type and compression levels during export
Building on the zip export improvements, this now allows users to further configure the
zip to fit their needs. A simple stored zip for speed, or a high compression zstd for
the smallest archive. Full validation of the method and levels at the command line
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Added a new --url argument to specify the base URL of the Paperless instance, allowing matched documents to be displayed as clickable links. Updated the logic to fetch document titles based on the presence of the base URL.
* Have the remote parser respect the provided produce_archive_file setting, as already determined via the consumer checks
* Updates the documentation to be correct about the respecting now
* merge conflict fixing
In tracemalloc based profiling, not materializing the whole Document list
reduced memory to approximately 20% of the baseline, with a peak memory
that scaled with the library size. Now, the lazt queryset is used and only
the needed pk value is actually contributing to memory
Trenton Holmes
shamoon
GitHub Actions
Max TruxaandGitHub
JaydenandGitHub