stumpylog
54be9dc9c5
Fix: finish manual and bulk zone OCR runs like other bulk edits
...
Runs reindexed each document separately, never sent document_updated
and did not bump modified for custom field writes. They now finish
through bulk_update_documents for the documents that got a value.
2026-11-19 14:18:30 -08:00
stumpylog
0bca50588d
Fix: drop a deleted custom field's combine format from OCR templates
...
The template kept the stale key, so saving it again failed validation.
2026-11-19 14:05:39 -08:00
stumpylog
246f4e53b1
Chore: describe the zone OCR responses in the API schema
...
The run status and skip reason were string literals, and the test zone
and 404 bodies had no schema. Add enums and response serializers for
them. The JSON is unchanged.
2026-11-19 13:54:59 -08:00
stumpylog
8f1a6d21ce
Chore: annotate the zone OCR views, signal and helpers
...
The OCR views, signal handler and regex helpers were untyped, which hid
dead path checks in the views. Annotate them and drop the dead checks.
2026-11-19 13:46:52 -08:00
stumpylog
d4f435de7d
Chore: type the OCR template serializers
...
Validated zone data and the serializer methods were untyped, so the
checkers could not follow zones through validation and saving. Add a
ZoneData TypedDict, annotate the methods and add serializer generics.
2026-11-19 13:36:48 -08:00
stumpylog
a9ad9d9d65
Chore: share one combine format key between OCR runs and validation
...
The run and the serializer each built the key and could drift. Both now
use field_key from zone_ocr.
2026-11-19 13:21:03 -08:00
stumpylog
ff6b5f3820
Chore: dispatch zone OCR on the target, transform and data type enums
...
The string if/elif chains ended in catch-alls. Match on the enums with
assert_never instead. A stored value outside the choices now raises,
failing only that template.
2026-11-19 12:41:40 -08:00
stumpylog
f35a2a4343
Chore: give zone OCR values a real type
...
Values were typed as object, and a result's value doubled as the
extracted text. Add a ZoneValue alias, carry the text separately, and
make the result dataclasses keyword-only.
2026-11-19 12:26:16 -08:00
stumpylog
a490d9d348
Chore: type zone OCR results and reasons
...
Reasons become a StrEnum, the run response has a serializer instead of an
untyped dict list, and the zone preview returns its dataclass. The JSON is
unchanged.
2026-11-19 10:15:43 -08:00
stumpylog
4c2df48795
Chore: give OCR templates their own matching fields
...
Templates are never assigned to documents, so the classifier-based Automatic
matching inherited from MatchingModel could never work. Declare the matching
fields with the workflow trigger choices instead. Also keep zones when their
custom field is deleted, allow longer OCR language lists, fix help texts,
and regenerate the unreleased migration.
2026-11-19 10:04:31 -08:00
stumpylog
8e07452604
Fix: reject malformed zone OCR language codes
...
Garbage values went straight to tesseract and looked like an empty read.
2026-11-19 09:12:37 -08:00
stumpylog
f81ab41c1b
Fix: reject zone date formats that cannot read a full date
...
A bad format silently fell back to auto-detection, and one without a year
read every date as 1900.
2026-11-19 09:09:40 -08:00
stumpylog
ee83aff920
Fix: require unique zone names and known combine format keys
...
Combine tokens refer to zones by name, and formats for fields no zone uses
were silently ignored.
2026-11-19 09:06:09 -08:00
stumpylog
6eb0e4e6b3
Fix: reject a custom field on zones with a built-in target
...
Title, ASN and created zones stored a custom field that was then ignored.
2026-11-19 09:02:40 -08:00
stumpylog
884fbf7fb2
Fix: reject invalid zone validation regexes on save
...
A bad pattern made every extraction fail the regex check.
2026-11-19 09:00:05 -08:00
stumpylog
0f4a71ed88
Fix: let documents without an owner use every OCR template
...
Unowned documents only matched unowned templates, so they never matched
templates created in the UI. Consider every template, as matching does.
2026-11-19 08:43:29 -08:00
stumpylog
0117fed721
Fix: limit OCR template lookups to objects the user can see
...
Template and document references gave 400 for a missing id but 403 for a
hidden one, and a template could point at a hidden sample document. Narrow
the querysets, keeping a template's stored sample document, and drop the
view checks they replace.
2026-11-19 08:40:49 -08:00
stumpylog
f401267c4b
Fix: let template viewers preview zones
...
The zone preview is a POST, so it required permission to add templates. Both
preview endpoints now require viewing templates and documents.
2026-11-19 08:27:51 -08:00
stumpylog
0e7cbda3d9
Fix: don't pass raw text on when a zone transform fails
...
A date or numeric transform that found nothing returned the OCR text, so the
field got junk. Return nothing and report transform_failed, which also blocks
a combined field.
2026-11-19 08:19:07 -08:00
stumpylog
fcc465ec2e
Fix: reject over-long text zone values instead of truncating
...
A truncated URL is a different URL. Let field validation reject string and URL
values that are too long.
2026-11-19 08:10:35 -08:00
stumpylog
7b25551e1e
Fix: drop only the ASN when a zone's ASN is already in use
...
A duplicate ASN failed the save and rolled back every write of the template.
Check first and report the zone as duplicate_asn instead.
2026-11-19 08:06:08 -08:00
stumpylog
3504ad07c6
Fix: keep zone number parsing linear and cap its input
...
Two patterns backtracked quadratically on long OCR text, taking seconds on a
large zone. Rewrite both, skip numbers and amounts longer than 256 characters,
and search only the first 256 characters for a date.
2026-11-19 08:02:51 -08:00
stumpylog
6fc4526947
Fix: keep the currency and validate monetary zone values
...
OCR dropped currency codes and stored amounts the API would reject. Keep an
ISO code or locale currency symbol, read parentheses as negative, and validate
with the same rule as the API.
2026-11-19 07:34:16 -08:00
stumpylog
218122452c
Chore: test zone OCR outcomes instead of mocking internals
...
Run real pdftoppm and tesseract on a sample page, and stop patching
_process_template in the extraction pipeline tests.
2026-11-19 07:21:28 -08:00
shamoon
4e2bda3a35
Merge branch 'dev' into feature/ocr-templates
2026-11-18 16:30:55 -08:00
shamoon
14252054a1
Fix/tweak: update form inputs while typing ( #14821 )
2026-11-18 16:28:02 -08:00
shamoon
61c8eec856
Fix page count, picker corners
2026-11-18 15:11:13 -08:00
stumpylog
409539d4d1
Chore: render zone OCR pages through the shared pdftoppm helper
...
pdftoppm was called in three places with different options. Missing-binary handling is dropped since pdftoppm and tesseract are required at startup.
2026-11-18 14:51:40 -08:00
stumpylog
c1bb327f1f
Fix: keep OCR values intact when combining zones into one field
...
Separator cleanup ran over the extracted values too, so a negative amount lost its sign. A zone failing validation also no longer lets a partial value through.
2026-11-18 14:20:11 -08:00
stumpylog
0e6d06c255
Feature: find dates after a label in OCR zones
...
A crop may include a label, such as "Datum: 13.04.2026", and those dates never parsed unless they were ISO.
2026-11-18 13:56:25 -08:00
stumpylog
acf7ba346c
Fix: parse OCR zone dates strictly using the zone's language
...
Dates ignored the document's day/month order, and words like "now" or a bare "5" were accepted as dates.
2026-11-18 13:46:21 -08:00
stumpylog
63a87dd11b
Fix: parse OCR numbers with Babel using the zone's language
...
Separators were guessed from the text, so "12.50" became 1250 for integer fields and the thousands and decimal separators were often swapped.
2026-11-18 13:34:26 -08:00
stumpylog
1f41e63af1
Chore: share compile and timeout handling across the safe regex helpers
2026-11-18 13:09:37 -08:00
stumpylog
dc16714e19
Chore: align zone OCR test naming, URLs and docstrings
2026-11-18 13:03:41 -08:00
stumpylog
044da9a627
Chore: move misplaced zone OCR tests
2026-11-18 12:55:53 -08:00
stumpylog
093c18e68b
Chore: parametrize near-duplicate zone OCR tests
2026-11-18 12:52:03 -08:00
stumpylog
d706904a18
Chore: remove duplicate zone OCR tests and assert on untested ones
2026-11-18 12:48:20 -08:00
stumpylog
1c6742723f
Chore: use shared user and client fixtures in zone OCR tests
2026-11-18 12:45:11 -08:00
stumpylog
e03faab730
Chore: share zone OCR mock fixtures and helpers
2026-11-18 12:41:46 -08:00
stumpylog
9ff48ecfce
Chore: add CustomFieldFactory
2026-11-18 12:35:38 -08:00
stumpylog
65fdfcdea7
Chore: default OcrTemplateFactory to MATCH_ANY and drop redundant test arguments
...
Match the model's default matching, so tests stop restating it along with other factory defaults.
2026-11-18 12:33:49 -08:00
stumpylog
bb78373ad1
Chore: move zone OCR bulk edit API tests to a pytest class
2026-11-18 08:35:33 -08:00
stumpylog
73e0390a84
Chore: group zone OCR signal tests
2026-11-18 08:33:13 -08:00
stumpylog
6009027bf7
Chore: group zone OCR bulk operation tests and use mocker
2026-11-18 08:31:24 -08:00
stumpylog
17cdcda76b
Chore: convert OCR template preview API tests to pytest
2026-11-18 08:28:50 -08:00
stumpylog
88d0f30c1a
Chore: convert run zone OCR API tests to pytest
2026-11-18 08:23:48 -08:00
stumpylog
e3904fe376
Chore: convert OCR template CRUD API tests to pytest
2026-11-18 08:19:10 -08:00
stumpylog
9d5c7abfaa
Chore: convert zone OCR database tests to pytest
...
TestRunZoneExtraction used the real scratch dir, so it failed when run on its own.
2026-11-18 08:10:27 -08:00
stumpylog
1c685eb85a
Chore: convert zone OCR extraction and render tests to pytest
2026-11-18 08:02:23 -08:00
stumpylog
44bbfd8932
Chore: convert zone OCR value function tests to pytest
2026-11-18 07:57:14 -08:00