Commit Graph
2632 Commits
Author SHA1 Message Date
stumpylog 884fbf7fb2 Fix: reject invalid zone validation regexes on save
A bad pattern made every extraction fail the regex check.
2026-11-19 09:00:05 -08:00
stumpylog 0f4a71ed88 Fix: let documents without an owner use every OCR template
Unowned documents only matched unowned templates, so they never matched
templates created in the UI. Consider every template, as matching does.
2026-11-19 08:43:29 -08:00
stumpylog 0117fed721 Fix: limit OCR template lookups to objects the user can see
Template and document references gave 400 for a missing id but 403 for a
hidden one, and a template could point at a hidden sample document. Narrow
the querysets, keeping a template's stored sample document, and drop the
view checks they replace.
2026-11-19 08:40:49 -08:00
stumpylog f401267c4b Fix: let template viewers preview zones
The zone preview is a POST, so it required permission to add templates. Both
preview endpoints now require viewing templates and documents.
2026-11-19 08:27:51 -08:00
stumpylog 0e7cbda3d9 Fix: don't pass raw text on when a zone transform fails
A date or numeric transform that found nothing returned the OCR text, so the
field got junk. Return nothing and report transform_failed, which also blocks
a combined field.
2026-11-19 08:19:07 -08:00
stumpylog fcc465ec2e Fix: reject over-long text zone values instead of truncating
A truncated URL is a different URL. Let field validation reject string and URL
values that are too long.
2026-11-19 08:10:35 -08:00
stumpylog 7b25551e1e Fix: drop only the ASN when a zone's ASN is already in use
A duplicate ASN failed the save and rolled back every write of the template.
Check first and report the zone as duplicate_asn instead.
2026-11-19 08:06:08 -08:00
stumpylog 3504ad07c6 Fix: keep zone number parsing linear and cap its input
Two patterns backtracked quadratically on long OCR text, taking seconds on a
large zone. Rewrite both, skip numbers and amounts longer than 256 characters,
and search only the first 256 characters for a date.
2026-11-19 08:02:51 -08:00
stumpylog 6fc4526947 Fix: keep the currency and validate monetary zone values
OCR dropped currency codes and stored amounts the API would reject. Keep an
ISO code or locale currency symbol, read parentheses as negative, and validate
with the same rule as the API.
2026-11-19 07:34:16 -08:00
stumpylog 218122452c Chore: test zone OCR outcomes instead of mocking internals
Run real pdftoppm and tesseract on a sample page, and stop patching
_process_template in the extraction pipeline tests.
2026-11-19 07:21:28 -08:00
stumpylog 409539d4d1 Chore: render zone OCR pages through the shared pdftoppm helper
pdftoppm was called in three places with different options. Missing-binary handling is dropped since pdftoppm and tesseract are required at startup.
2026-11-18 14:51:40 -08:00
stumpylog c1bb327f1f Fix: keep OCR values intact when combining zones into one field
Separator cleanup ran over the extracted values too, so a negative amount lost its sign. A zone failing validation also no longer lets a partial value through.
2026-11-18 14:20:11 -08:00
stumpylog 0e6d06c255 Feature: find dates after a label in OCR zones
A crop may include a label, such as "Datum: 13.04.2026", and those dates never parsed unless they were ISO.
2026-11-18 13:56:25 -08:00
stumpylog acf7ba346c Fix: parse OCR zone dates strictly using the zone's language
Dates ignored the document's day/month order, and words like "now" or a bare "5" were accepted as dates.
2026-11-18 13:46:21 -08:00
stumpylog 63a87dd11b Fix: parse OCR numbers with Babel using the zone's language
Separators were guessed from the text, so "12.50" became 1250 for integer fields and the thousands and decimal separators were often swapped.
2026-11-18 13:34:26 -08:00
stumpylog 1f41e63af1 Chore: share compile and timeout handling across the safe regex helpers 2026-11-18 13:09:37 -08:00
stumpylog dc16714e19 Chore: align zone OCR test naming, URLs and docstrings 2026-11-18 13:03:41 -08:00
stumpylog 044da9a627 Chore: move misplaced zone OCR tests 2026-11-18 12:55:53 -08:00
stumpylog 093c18e68b Chore: parametrize near-duplicate zone OCR tests 2026-11-18 12:52:03 -08:00
stumpylog d706904a18 Chore: remove duplicate zone OCR tests and assert on untested ones 2026-11-18 12:48:20 -08:00
stumpylog 1c6742723f Chore: use shared user and client fixtures in zone OCR tests 2026-11-18 12:45:11 -08:00
stumpylog e03faab730 Chore: share zone OCR mock fixtures and helpers 2026-11-18 12:41:46 -08:00
stumpylog 9ff48ecfce Chore: add CustomFieldFactory 2026-11-18 12:35:38 -08:00
stumpylog 65fdfcdea7 Chore: default OcrTemplateFactory to MATCH_ANY and drop redundant test arguments
Match the model's default matching, so tests stop restating it along with other factory defaults.
2026-11-18 12:33:49 -08:00
stumpylog bb78373ad1 Chore: move zone OCR bulk edit API tests to a pytest class 2026-11-18 08:35:33 -08:00
stumpylog 73e0390a84 Chore: group zone OCR signal tests 2026-11-18 08:33:13 -08:00
stumpylog 6009027bf7 Chore: group zone OCR bulk operation tests and use mocker 2026-11-18 08:31:24 -08:00
stumpylog 17cdcda76b Chore: convert OCR template preview API tests to pytest 2026-11-18 08:28:50 -08:00
stumpylog 88d0f30c1a Chore: convert run zone OCR API tests to pytest 2026-11-18 08:23:48 -08:00
stumpylog e3904fe376 Chore: convert OCR template CRUD API tests to pytest 2026-11-18 08:19:10 -08:00
stumpylog 9d5c7abfaa Chore: convert zone OCR database tests to pytest
TestRunZoneExtraction used the real scratch dir, so it failed when run on its own.
2026-11-18 08:10:27 -08:00
stumpylog 1c685eb85a Chore: convert zone OCR extraction and render tests to pytest 2026-11-18 08:02:23 -08:00
stumpylog 44bbfd8932 Chore: convert zone OCR value function tests to pytest 2026-11-18 07:57:14 -08:00
stumpylog 4788ac602b Fix: limit manually run OCR templates to those the requester can view
A user editing someone else's document could run the owner's private
templates and see their names. Manual runs now use templates both can view.
2026-11-18 07:55:52 -08:00
stumpylog 0c90c57e3e Fix: include OCR templates in the document exporter
Templates and their zones were missing from the export manifest, so an
import lost them.
2026-11-18 07:55:52 -08:00
shamoon 855b161407 Move run_zone_ocr to the new bulk operations pathway 2026-11-17 13:43:13 -08:00
shamoon 3153b118bd Merge branch 'dev' into feature/ocr-templates 2026-11-17 13:30:29 -08:00
shamoon 237521ec4b Fix: skip dont use set_actor if audit log disabled (#14808) 2026-11-17 21:02:16 +00:00
Trenton H 199d1a88e9 Fix: stream AI chat answers under ASGI (#14803) 2026-11-17 20:47:55 +00:00
shamoon de75249e51 Chorehancement: move all bulk operations to async hand-off (#14773) 2026-11-17 00:23:44 +00:00
Trenton H c177c13acf Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters (#14783)
* Fix: use a PostgreSQL-compatible regex for scheduled workflow filename filters

* Tests for the translation layer
2026-11-16 06:42:34 -08:00
shamoon d31110278b Fix: ensure ownership for manual WF runs (#14762) 2026-11-13 21:13:34 +00:00
shamoon 30e95a6d28 Fix migration 2026-11-13 09:20:47 -08:00
shamoon 8c50a5ee7b Merge branch 'dev' into feature/ocr-templates 2026-11-13 09:09:39 -08:00
Johannes Hagerandshamoon b63ea77543 Enhancement: render document notes as markdown (#14626)
---------

Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
2026-11-13 09:08:20 -08:00
shamoon 3dbeca603e Feature: manual workflow trigger (#14704) 2026-11-12 13:28:10 -08:00
shamoon 60a35edee1 Enhancement: make bulk metadata operations run async (#14711) 2026-11-11 15:50:15 -08:00
Trenton H d05e96cda2 Fix: Blacklist certain OCR user arguments that paperless itself controls/uses (#14734) 2026-11-11 03:13:20 +00:00
dependabot[bot]andstumpylog d7d4014bbd Chore(deps): Bump the pre-commit-dependencies group with 4 updates (#14636)
* Chore(deps): Bump the pre-commit-dependencies group with 4 updates

Bumps the pre-commit-dependencies group with 4 updates: [https://github.com/rbubley/mirrors-prettier](https://github.com/rbubley/mirrors-prettier), prettier, [https://github.com/astral-sh/ruff-pre-commit](https://github.com/astral-sh/ruff-pre-commit) and [https://github.com/tox-dev/pyproject-fmt](https://github.com/tox-dev/pyproject-fmt).


Updates `https://github.com/rbubley/mirrors-prettier` from v3.9.9 to 3.9.10
- [Commits](https://github.com/rbubley/mirrors-prettier/compare/v3.9.9...v3.9.10)

Updates `prettier` from 3.9.9 to 3.9.10

Updates `https://github.com/astral-sh/ruff-pre-commit` from v0.16.9 to 0.17.2
- [Release notes](https://github.com/astral-sh/ruff-pre-commit/releases)
- [Commits](https://github.com/astral-sh/ruff-pre-commit/compare/v0.16.9...v0.17.2)

Updates `https://github.com/tox-dev/pyproject-fmt` from v2.29.4 to 2.30.3
- [Release notes](https://github.com/tox-dev/pyproject-fmt/releases)
- [Commits](https://github.com/tox-dev/pyproject-fmt/compare/v2.29.4...v2.30.3)

---
updated-dependencies:
- dependency-name: https://github.com/rbubley/mirrors-prettier
  dependency-version: 3.9.10
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
- dependency-name: prettier
  dependency-version: 3.9.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: pre-commit-dependencies
- dependency-name: https://github.com/astral-sh/ruff-pre-commit
  dependency-version: 0.17.2
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
- dependency-name: https://github.com/tox-dev/pyproject-fmt
  dependency-version: 2.30.3
  dependency-type: direct:production
  dependency-group: pre-commit-dependencies
...

Signed-off-by: dependabot[bot] <support@github.com>

* Linting fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-11-10 17:49:06 +00:00
shamoon a139bdff46 Fixhancement: add automatic clearing of sessions (#14722) 2026-11-10 08:09:25 -08:00