Commit Graph
12701 Commits
Author SHA1 Message Date
stumpylog a7d4c04a4b Chore: test the bulk zone OCR permission checks 2026-11-20 12:51:02 -08:00
stumpylog dcb0d2014a Chore: test that OCR templates cannot be deleted without permission 2026-11-20 12:49:40 -08:00
stumpylog 499ed8d442 Chore: keep the OCR field type list next to the converter 2026-11-20 11:54:30 -08:00
stumpylog 2d5e419850 Chore: match a root's own content without re-fetching the document 2026-11-20 11:46:02 -08:00
stumpylog 4d714f263b Chore: drop dead code left in zone OCR 2026-11-20 11:39:08 -08:00
stumpylog 17d5cb8855 Fix: let the first matching OCR template win a field
Later templates no longer overwrite it. The skipped zones report superseded and a warning is logged.
2026-11-20 11:30:31 -08:00
stumpylog a2c18123fa Fix: convert crops PNG cannot store before handing them to tesseract 2026-11-20 11:24:06 -08:00
stumpylog ad5a7b365e Fix: apply EXIF orientation when cropping zone images 2026-11-20 11:20:10 -08:00
stumpylog a54141058d Chore: number the editor page image URL from 1 like zone pages 2026-11-20 10:05:14 -08:00
stumpylog e81bdff6c2 Fix: report a last-page zone as missing when the page count is unknown
An image counts as one page and a PDF without a stored count is counted from the file.
2026-11-20 09:59:52 -08:00
stumpylog 6b8b946567 Chore: document the page numbering in zone OCR 2026-11-20 09:52:58 -08:00
stumpylog ad64f2a1b1 Chore: import dateparser and zxingcpp lazily in zone OCR 2026-11-20 09:51:57 -08:00
stumpylog 0348dc532f Fix: create the scratch dir before zone OCR renders into it 2026-11-20 09:38:50 -08:00
stumpylog 5182b719c5 Chore: resolve the zone OCR source file in one place 2026-11-20 09:34:24 -08:00
stumpylog 36de14d2c5 Fix: read the effective document version in the zone editor endpoints
The page image and test-zone endpoints now resolve the version like zone OCR runs do, including ?version=.
2026-11-20 09:28:36 -08:00
stumpylog f0c91bbd59 Chore: pass is_image to zone OCR instead of sniffing the file type 2026-11-20 09:20:25 -08:00
stumpylog ccbff94208 Fix: make bulk zone OCR survive a missing template or failing document
Inline extract_zones into run_zone_ocr, load documents and the template once, and process each root once. A failing document is logged and the task fails after the rest finish.
2026-11-20 09:09:28 -08:00
shamoon a08918eb9c Update frontend to use the bulk operation queuing pathway 2026-11-19 22:25:07 -08:00
stumpylog c5fdba73a1 Fix: queue single document zone OCR runs instead of running them in the request
Rendering and OCR ran inside the HTTP request and could block other
requests for minutes. The action now validates, then queues the bulk
zone OCR operation for the selected file and returns its task id.
2026-11-19 15:25:48 -08:00
stumpylog a6b81b2d64 Fix: read zones before the consume transaction and write them before workflows
Zone OCR ran inside the consume transaction, read the working copy instead
of the archive, and ran after the document added workflows. Zones are now
read from the archive before the transaction and written just before
those workflows, keeping values supplied at consume time.
2026-11-19 15:09:41 -08:00
stumpylog 54be9dc9c5 Fix: finish manual and bulk zone OCR runs like other bulk edits
Runs reindexed each document separately, never sent document_updated
and did not bump modified for custom field writes. They now finish
through bulk_update_documents for the documents that got a value.
2026-11-19 14:18:30 -08:00
stumpylog 0bca50588d Fix: drop a deleted custom field's combine format from OCR templates
The template kept the stale key, so saving it again failed validation.
2026-11-19 14:05:39 -08:00
stumpylog 246f4e53b1 Chore: describe the zone OCR responses in the API schema
The run status and skip reason were string literals, and the test zone
and 404 bodies had no schema. Add enums and response serializers for
them. The JSON is unchanged.
2026-11-19 13:54:59 -08:00
stumpylog 8f1a6d21ce Chore: annotate the zone OCR views, signal and helpers
The OCR views, signal handler and regex helpers were untyped, which hid
dead path checks in the views. Annotate them and drop the dead checks.
2026-11-19 13:46:52 -08:00
stumpylog d4f435de7d Chore: type the OCR template serializers
Validated zone data and the serializer methods were untyped, so the
checkers could not follow zones through validation and saving. Add a
ZoneData TypedDict, annotate the methods and add serializer generics.
2026-11-19 13:36:48 -08:00
stumpylog a9ad9d9d65 Chore: share one combine format key between OCR runs and validation
The run and the serializer each built the key and could drift. Both now
use field_key from zone_ocr.
2026-11-19 13:21:03 -08:00
stumpylog ff6b5f3820 Chore: dispatch zone OCR on the target, transform and data type enums
The string if/elif chains ended in catch-alls. Match on the enums with
assert_never instead. A stored value outside the choices now raises,
failing only that template.
2026-11-19 12:41:40 -08:00
stumpylog f35a2a4343 Chore: give zone OCR values a real type
Values were typed as object, and a result's value doubled as the
extracted text. Add a ZoneValue alias, carry the text separately, and
make the result dataclasses keyword-only.
2026-11-19 12:26:16 -08:00
stumpylog a490d9d348 Chore: type zone OCR results and reasons
Reasons become a StrEnum, the run response has a serializer instead of an
untyped dict list, and the zone preview returns its dataclass. The JSON is
unchanged.
2026-11-19 10:15:43 -08:00
stumpylog 4c2df48795 Chore: give OCR templates their own matching fields
Templates are never assigned to documents, so the classifier-based Automatic
matching inherited from MatchingModel could never work. Declare the matching
fields with the workflow trigger choices instead. Also keep zones when their
custom field is deleted, allow longer OCR language lists, fix help texts,
and regenerate the unreleased migration.
2026-11-19 10:04:31 -08:00
stumpylog 8e07452604 Fix: reject malformed zone OCR language codes
Garbage values went straight to tesseract and looked like an empty read.
2026-11-19 09:12:37 -08:00
stumpylog f81ab41c1b Fix: reject zone date formats that cannot read a full date
A bad format silently fell back to auto-detection, and one without a year
read every date as 1900.
2026-11-19 09:09:40 -08:00
stumpylog ee83aff920 Fix: require unique zone names and known combine format keys
Combine tokens refer to zones by name, and formats for fields no zone uses
were silently ignored.
2026-11-19 09:06:09 -08:00
stumpylog 6eb0e4e6b3 Fix: reject a custom field on zones with a built-in target
Title, ASN and created zones stored a custom field that was then ignored.
2026-11-19 09:02:40 -08:00
stumpylog 884fbf7fb2 Fix: reject invalid zone validation regexes on save
A bad pattern made every extraction fail the regex check.
2026-11-19 09:00:05 -08:00
stumpylog 0f4a71ed88 Fix: let documents without an owner use every OCR template
Unowned documents only matched unowned templates, so they never matched
templates created in the UI. Consider every template, as matching does.
2026-11-19 08:43:29 -08:00
stumpylog 0117fed721 Fix: limit OCR template lookups to objects the user can see
Template and document references gave 400 for a missing id but 403 for a
hidden one, and a template could point at a hidden sample document. Narrow
the querysets, keeping a template's stored sample document, and drop the
view checks they replace.
2026-11-19 08:40:49 -08:00
stumpylog f401267c4b Fix: let template viewers preview zones
The zone preview is a POST, so it required permission to add templates. Both
preview endpoints now require viewing templates and documents.
2026-11-19 08:27:51 -08:00
stumpylog 0e7cbda3d9 Fix: don't pass raw text on when a zone transform fails
A date or numeric transform that found nothing returned the OCR text, so the
field got junk. Return nothing and report transform_failed, which also blocks
a combined field.
2026-11-19 08:19:07 -08:00
stumpylog fcc465ec2e Fix: reject over-long text zone values instead of truncating
A truncated URL is a different URL. Let field validation reject string and URL
values that are too long.
2026-11-19 08:10:35 -08:00
stumpylog 7b25551e1e Fix: drop only the ASN when a zone's ASN is already in use
A duplicate ASN failed the save and rolled back every write of the template.
Check first and report the zone as duplicate_asn instead.
2026-11-19 08:06:08 -08:00
stumpylog 3504ad07c6 Fix: keep zone number parsing linear and cap its input
Two patterns backtracked quadratically on long OCR text, taking seconds on a
large zone. Rewrite both, skip numbers and amounts longer than 256 characters,
and search only the first 256 characters for a date.
2026-11-19 08:02:51 -08:00
stumpylog 6fc4526947 Fix: keep the currency and validate monetary zone values
OCR dropped currency codes and stored amounts the API would reject. Keep an
ISO code or locale currency symbol, read parentheses as negative, and validate
with the same rule as the API.
2026-11-19 07:34:16 -08:00
stumpylog 218122452c Chore: test zone OCR outcomes instead of mocking internals
Run real pdftoppm and tesseract on a sample page, and stop patching
_process_template in the extraction pipeline tests.
2026-11-19 07:21:28 -08:00
shamoon 4e2bda3a35 Merge branch 'dev' into feature/ocr-templates 2026-11-18 16:30:55 -08:00
shamoon 14252054a1 Fix/tweak: update form inputs while typing (#14821) 2026-11-18 16:28:02 -08:00
shamoon 61c8eec856 Fix page count, picker corners 2026-11-18 15:11:13 -08:00
stumpylog 409539d4d1 Chore: render zone OCR pages through the shared pdftoppm helper
pdftoppm was called in three places with different options. Missing-binary handling is dropped since pdftoppm and tesseract are required at startup.
2026-11-18 14:51:40 -08:00
stumpylog c1bb327f1f Fix: keep OCR values intact when combining zones into one field
Separator cleanup ran over the extracted values too, so a negative amount lost its sign. A zone failing validation also no longer lets a partial value through.
2026-11-18 14:20:11 -08:00
stumpylog 0e6d06c255 Feature: find dates after a label in OCR zones
A crop may include a label, such as "Datum: 13.04.2026", and those dates never parsed unless they were ISO.
2026-11-18 13:56:25 -08:00