Commit Graph
37 Commits
Author SHA1 Message Date
Trenton HolmesandClaude Fable 5 7bab9622c8 fix(search): restore unquoted multi-word date keywords via pre-parse quoting
"added:previous month" returned HTTP 400 after the whoosh-compat
migration. The unquoted spelling was never parser-native anywhere: v2
rewrote it to explicit bracket ranges app-side before whoosh saw the
string, and the deleted translation layer consumed it itself, so users
and saved views have relied on it continuously while whoosh-compat
deliberately scopes it out of its parser (its DIVERGENCES.md entry 19)
and understands the phrases natively only as quoted values.

parse_user_query now quotes the closed six-phrase vocabulary (previous
week/month/quarter/year, this month/year) when it directly follows a
date field's colon, before parsing. Only quoting happens app-side; every
date computation stays in whoosh-compat's grammar, unlike v2's rewrite,
which computed the ranges itself. Date field names derive from
PUBLIC_FIELDS, the field name matches case-sensitively (the parser's own
field tagging is case-sensitive), the phrase case-insensitively (the
grammar accepts any case in the quoted form), and already-quoted
spellings, TEXT fields, unfielded words and bracketed ranges are
untouched.

The previously xfailed end-to-end regression test now passes as a plain
test, and a new acceptance class pins unquoted == quoted == mixed-case
result sets on a boundary fixture, no-error parsing for the whole
vocabulary across all three date fields, and that "title:previous month"
stays an ordinary text search. docs/usage.md now states the two
spellings are equivalent after a date field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WMsn6DgzbvSqh1pwy66VVF
2026-08-18 11:05:04 -07:00
Trenton Holmes 0bf06f89a7 refactor: minor cleanup from final whoosh-compat migration review
- Update stale test comments in test_query.py that described string
  rewriting / raw-query fallback behavior that no longer exists post
  whoosh-compat migration; rename
  test_date_rewriting_applied_before_tantivy_parse to
  test_date_keyword_resolves_without_raising to match.
- views.py: move the local MultipleSearchQueryErrors import up into the
  existing local-import block near the top of list(), consistent with
  the other documents.search imports there, instead of importing it
  again inside the except SearchQueryError clause.
- test_api_search.py: assert response.status_code explicitly before
  indexing into response.data["results"] in
  test_search_added_previous_month_excludes_next_period_start, and tie
  the xfail marker to AssertionError instead of the incidental KeyError
  that indexing a 400 response's missing "results" key produced.
2026-08-18 11:05:04 -07:00
Trenton Holmes 5942cfb860 test(api): add end-to-end search coverage for asn/page_count/original_filename/checksum 2026-08-18 11:05:04 -07:00
Trenton HolmesandClaude Sonnet 5 bb157726c9 feat(api): surface every search query error, not just the first
When parse_user_query() raises MultipleSearchQueryErrors due to multiple
field parsing failures (e.g. both an invalid date and an invalid number
in a single query), the exception handler now surfaces all error messages
in the 400 response, allowing users to fix them all in one round-trip
instead of discovering them one at a time.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-18 11:05:04 -07:00
Trenton HolmesandClaude Sonnet 5 d7ccff138b feat(search): route parse_user_query through whoosh-compat
Rewires parse_user_query() to parse via wc.parse()/tantivy_emit() against
the shared FieldRegistry instead of the string-based translate_query()
pipeline, so diagnostics map to typed SearchQueryError subclasses
(InvalidDateQuery/InvalidNumberQuery/MultipleSearchQueryErrors) and every
bad field is reported, not just the first.

Marks three pre-existing tests xfail (2 in test_query.py, 1 in
test_api_search.py) for confirmed whoosh-compat grammar gaps found while
verifying this rewrite: unquoted multi-word date keywords (e.g.
`added:previous month`) and RFC3339 T/Z datetime range bounds no longer
parse.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-08-18 11:05:04 -07:00
Trenton HandGitHub 5ad34dfe03 Fix: Set the document modified dates to hopefully prevent this flake. Good across 30 runs at least (#13584) 2026-08-07 01:06:02 +00:00
Trenton HandGitHub 1b32b9d678 Fix: exclude next-period start from relative date-range filters (#13381)
* Fix: exclude next-period start from relative date-range filters

Tantivy's [lo TO hi] range is inclusive on both ends, but computed upper
bounds (keyword ranges, YYYY/YYYYMM/YYYYMMDD tokens) represent the start of
the next period. Use half-open [lo TO hi} for those so e.g. "previous month"
no longer matches the 1st of the current month.

* Adds a regression test down to the second check for the hi range
2026-07-28 16:58:38 +00:00
81795bc93a Security (beta): enforce current permissions in autocomplete (#13188)
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-22 08:07:43 -07:00
f4fa916579 Fix (beta): restore v2 (Whoosh) advanced-search query compatibility (#13010)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-15 15:32:44 -07:00
Trenton HandGitHub bbceb5dac6 Fix: Don't store autocomplete_word, only index it (#12867) 2026-05-29 14:09:04 -07:00
shamoonandGitHub a76b6b826c Fix (dev): resolve tantivy search-filtered documents in bulk edit (#12705) 2026-05-04 14:34:08 -07:00
Trenton HandGitHub 14fe520319 Chore: Update typing and baselines again (#12641)
a
2026-04-28 09:28:05 -07:00
shamoonandGitHub 20aa0937e8 Fix (dev): retain backwards compatibility with natural-date keywords in tantivy (#12602) 2026-04-20 08:26:33 -07:00
3ffbb8862c Feature: paginate search highlights and remove 10k document search limit (#12518)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-04-15 23:20:31 +00:00
shamoonandGitHub 566afdffca Enhancement: unify text search to use tantivy (#12485) 2026-04-03 13:53:45 -07:00
aed9abe48c Feature: Replace Whoosh with tantivy search backend (#12471)
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Antoine Mérino <3023499+Merinorus@users.noreply.github.com>
2026-04-02 12:38:22 -07:00
shamoonandGitHub 245514ad10 Performance: deprecate and remove usage of all in API results (#12309) 2026-03-31 07:55:59 -07:00
shamoonandGitHub f715533770 Performance: support passing selection data with filtered document requests (#12300) 2026-03-30 16:38:52 +00:00
shamoon 0f84af27d0 Merge branch 'main' into dev
# Conflicts:
#	docs/setup.md
#	src-ui/src/main.ts
#	src/documents/tests/test_api_bulk_edit.py
#	src/documents/tests/test_api_custom_fields.py
#	src/documents/tests/test_api_search.py
#	src/documents/tests/test_api_status.py
#	src/documents/tests/test_workflows.py
#	src/paperless_mail/tests/test_api.py
2026-03-21 02:12:19 -07:00
shamoon cc71aad058 Fix: suggest corrections only if visible results 2026-03-21 01:24:23 -07:00
shamoon 3cbdf5d0b7 Fix: require view permission for more-like search 2026-03-21 01:20:59 -07:00
shamoon 9e9fc6213c Resolve GHSA-96jx-fj7m-qh6x 2026-03-20 15:39:15 -07:00
shamoonandGitHub 85a18e5911 Enhancement: saved view sharing (#12142) 2026-03-04 14:15:43 -08:00
Sebastian SteinbeißerandGitHub 3b5ffbf9fa Chore(mypy): Annotate None returns for typing improvements (#11213) 2026-02-02 08:44:12 -08:00
shamoonandGitHub 128c3539d5 Chore: fix set_permissions_for_object type (#11564) 2025-12-10 00:12:40 +00:00
shamoonandGitHub 0e5ab7f3e0 Fix: support for custom field ordering w advanced search (#11383) 2025-11-17 20:47:55 +00:00
shamoonandGitHub 51e6eed72a Fix: handle whoosh query correction errors (#10121) 2025-06-05 08:57:25 -07:00
shamoonandGitHub 1a6f32534c Change: treat created as date not datetime (#9793) 2025-05-16 14:23:04 +00:00
dependabot[bot]GitHubdependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>shamoon
20ec8cb57b Chore(deps-dev): Bump the development group with 2 updates (#8841)
* Chore(deps-dev): Bump the development group with 2 updates

Bumps the development group with 2 updates: [ruff](https://github.com/astral-sh/ruff) and [mkdocs-material](https://github.com/squidfunk/mkdocs-material).


Updates `ruff` from 0.8.6 to 0.9.2
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.8.6...0.9.2)

Updates `mkdocs-material` from 9.5.49 to 9.5.50
- [Release notes](https://github.com/squidfunk/mkdocs-material/releases)
- [Changelog](https://github.com/squidfunk/mkdocs-material/blob/master/CHANGELOG)
- [Commits](https://github.com/squidfunk/mkdocs-material/compare/9.5.49...9.5.50)

---
updated-dependencies:
- dependency-name: ruff
  dependency-type: direct:development
  update-type: version-update:semver-minor
  dependency-group: development
- dependency-name: mkdocs-material
  dependency-type: direct:development
  update-type: version-update:semver-patch
  dependency-group: development
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update .pre-commit-config.yaml

* Run new ruff format

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2025-01-21 19:22:25 +00:00
Yichi YangandGitHub a0c227fe55 Refactor: Use django-filter logic for filtering full text search queries (#7507) 2024-08-24 21:20:43 -07:00
shamoonandGitHub c6e7d06bb7 Feature: global search, keyboard shortcuts / hotkey support (#6449) 2024-05-02 16:15:56 +00:00
shamoonandGitHub 63e1f9f5d3 Feature: custom fields filtering & bulk editing (#6484) 2024-04-26 15:10:03 -07:00
shamoonandGitHub fc68f79cc8 Enhancement: always place search term first in autocomplete results (#6142) 2024-03-21 19:03:17 +00:00
58bf9c552b Documentation: Fix typos with automated tool (#5319)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2024-01-08 16:58:41 +00:00
shamoonandGitHub 5e8de4c1da Enhancement: shared icon & shared by me filter (#4859) 2023-12-19 20:45:04 +00:00
Adam BogdałandGitHub 4510902677 Fix: Don't attempt to parse none objects during date searching 2023-12-14 07:39:49 -08:00
Trenton HandGitHub 7e12bd1bef Fix: Removes the FieldParser plugin from autocomplete searches (#4934) 2023-12-11 10:21:58 -08:00