Commit Graph
48 Commits
Author SHA1 Message Date
stumpylog d6ae3e1ec6 Chore: log a warning when a zone title is cut to 128 characters 2026-11-20 12:54:28 -08:00
stumpylog 499ed8d442 Chore: keep the OCR field type list next to the converter 2026-11-20 11:54:30 -08:00
stumpylog 2d5e419850 Chore: match a root's own content without re-fetching the document 2026-11-20 11:46:02 -08:00
stumpylog 4d714f263b Chore: drop dead code left in zone OCR 2026-11-20 11:39:08 -08:00
stumpylog 17d5cb8855 Fix: let the first matching OCR template win a field
Later templates no longer overwrite it. The skipped zones report superseded and a warning is logged.
2026-11-20 11:30:31 -08:00
stumpylog a2c18123fa Fix: convert crops PNG cannot store before handing them to tesseract 2026-11-20 11:24:06 -08:00
stumpylog ad5a7b365e Fix: apply EXIF orientation when cropping zone images 2026-11-20 11:20:10 -08:00
stumpylog e81bdff6c2 Fix: report a last-page zone as missing when the page count is unknown
An image counts as one page and a PDF without a stored count is counted from the file.
2026-11-20 09:59:52 -08:00
stumpylog 6b8b946567 Chore: document the page numbering in zone OCR 2026-11-20 09:52:58 -08:00
stumpylog ad64f2a1b1 Chore: import dateparser and zxingcpp lazily in zone OCR 2026-11-20 09:51:57 -08:00
stumpylog 0348dc532f Fix: create the scratch dir before zone OCR renders into it 2026-11-20 09:38:50 -08:00
stumpylog 5182b719c5 Chore: resolve the zone OCR source file in one place 2026-11-20 09:34:24 -08:00
stumpylog f0c91bbd59 Chore: pass is_image to zone OCR instead of sniffing the file type 2026-11-20 09:20:25 -08:00
stumpylog c5fdba73a1 Fix: queue single document zone OCR runs instead of running them in the request
Rendering and OCR ran inside the HTTP request and could block other
requests for minutes. The action now validates, then queues the bulk
zone OCR operation for the selected file and returns its task id.
2026-11-19 15:25:48 -08:00
stumpylog a6b81b2d64 Fix: read zones before the consume transaction and write them before workflows
Zone OCR ran inside the consume transaction, read the working copy instead
of the archive, and ran after the document added workflows. Zones are now
read from the archive before the transaction and written just before
those workflows, keeping values supplied at consume time.
2026-11-19 15:09:41 -08:00
stumpylog 54be9dc9c5 Fix: finish manual and bulk zone OCR runs like other bulk edits
Runs reindexed each document separately, never sent document_updated
and did not bump modified for custom field writes. They now finish
through bulk_update_documents for the documents that got a value.
2026-11-19 14:18:30 -08:00
stumpylog 246f4e53b1 Chore: describe the zone OCR responses in the API schema
The run status and skip reason were string literals, and the test zone
and 404 bodies had no schema. Add enums and response serializers for
them. The JSON is unchanged.
2026-11-19 13:54:59 -08:00
stumpylog 8f1a6d21ce Chore: annotate the zone OCR views, signal and helpers
The OCR views, signal handler and regex helpers were untyped, which hid
dead path checks in the views. Annotate them and drop the dead checks.
2026-11-19 13:46:52 -08:00
stumpylog a9ad9d9d65 Chore: share one combine format key between OCR runs and validation
The run and the serializer each built the key and could drift. Both now
use field_key from zone_ocr.
2026-11-19 13:21:03 -08:00
stumpylog ff6b5f3820 Chore: dispatch zone OCR on the target, transform and data type enums
The string if/elif chains ended in catch-alls. Match on the enums with
assert_never instead. A stored value outside the choices now raises,
failing only that template.
2026-11-19 12:41:40 -08:00
stumpylog f35a2a4343 Chore: give zone OCR values a real type
Values were typed as object, and a result's value doubled as the
extracted text. Add a ZoneValue alias, carry the text separately, and
make the result dataclasses keyword-only.
2026-11-19 12:26:16 -08:00
stumpylog a490d9d348 Chore: type zone OCR results and reasons
Reasons become a StrEnum, the run response has a serializer instead of an
untyped dict list, and the zone preview returns its dataclass. The JSON is
unchanged.
2026-11-19 10:15:43 -08:00
stumpylog 0f4a71ed88 Fix: let documents without an owner use every OCR template
Unowned documents only matched unowned templates, so they never matched
templates created in the UI. Consider every template, as matching does.
2026-11-19 08:43:29 -08:00
stumpylog 0e7cbda3d9 Fix: don't pass raw text on when a zone transform fails
A date or numeric transform that found nothing returned the OCR text, so the
field got junk. Return nothing and report transform_failed, which also blocks
a combined field.
2026-11-19 08:19:07 -08:00
stumpylog fcc465ec2e Fix: reject over-long text zone values instead of truncating
A truncated URL is a different URL. Let field validation reject string and URL
values that are too long.
2026-11-19 08:10:35 -08:00
stumpylog 7b25551e1e Fix: drop only the ASN when a zone's ASN is already in use
A duplicate ASN failed the save and rolled back every write of the template.
Check first and report the zone as duplicate_asn instead.
2026-11-19 08:06:08 -08:00
stumpylog 3504ad07c6 Fix: keep zone number parsing linear and cap its input
Two patterns backtracked quadratically on long OCR text, taking seconds on a
large zone. Rewrite both, skip numbers and amounts longer than 256 characters,
and search only the first 256 characters for a date.
2026-11-19 08:02:51 -08:00
stumpylog 6fc4526947 Fix: keep the currency and validate monetary zone values
OCR dropped currency codes and stored amounts the API would reject. Keep an
ISO code or locale currency symbol, read parentheses as negative, and validate
with the same rule as the API.
2026-11-19 07:34:16 -08:00
stumpylog 409539d4d1 Chore: render zone OCR pages through the shared pdftoppm helper
pdftoppm was called in three places with different options. Missing-binary handling is dropped since pdftoppm and tesseract are required at startup.
2026-11-18 14:51:40 -08:00
stumpylog c1bb327f1f Fix: keep OCR values intact when combining zones into one field
Separator cleanup ran over the extracted values too, so a negative amount lost its sign. A zone failing validation also no longer lets a partial value through.
2026-11-18 14:20:11 -08:00
stumpylog 0e6d06c255 Feature: find dates after a label in OCR zones
A crop may include a label, such as "Datum: 13.04.2026", and those dates never parsed unless they were ISO.
2026-11-18 13:56:25 -08:00
stumpylog acf7ba346c Fix: parse OCR zone dates strictly using the zone's language
Dates ignored the document's day/month order, and words like "now" or a bare "5" were accepted as dates.
2026-11-18 13:46:21 -08:00
stumpylog 63a87dd11b Fix: parse OCR numbers with Babel using the zone's language
Separators were guessed from the text, so "12.50" became 1250 for integer fields and the thousands and decimal separators were often swapped.
2026-11-18 13:34:26 -08:00
stumpylog 4788ac602b Fix: limit manually run OCR templates to those the requester can view
A user editing someone else's document could run the owner's private
templates and see their names. Manual runs now use templates both can view.
2026-11-18 07:55:52 -08:00
shamoon 886761a53b sonar 2026-11-09 09:32:40 -08:00
shamoon e63f27a712 Crap missed versioning stuff, default to latest but allow specific 2026-11-09 09:19:30 -08:00
shamoon 3ce5f8bc69 Ensure reindex happens after extraction on manual runs 2026-11-09 09:01:20 -08:00
shamoonandOpenAI Codex 54f7f43f35 Reuse OCR page renders and simplify value conversion
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:07:00 -07:00
shamoonandOpenAI Codex 94da3d4c57 Report actual OCR extraction outcomes instead of existing field values
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:02:23 -07:00
shamoonandOpenAI Codex 453e89d6dd Prepare and atomically apply OCR template field updates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:00:03 -07:00
shamoonandOpenAI Codex 6562415894 Unify OCR zone extraction for preview and saved templates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 16:58:10 -07:00
shamoon d2e6834602 Make sort order stable 2026-10-20 16:28:31 -07:00
shamoon bbdc016a32 Backend for manual tempalte choice 2026-10-20 13:51:54 -07:00
shamoon bedefd85f2 Remove the handlers, make the matching work 2026-10-20 13:39:09 -07:00
shamoon e976b2ec7a Tests 2026-10-20 11:23:44 -07:00
shamoon 05da0ba5a3 Apply only visible templates 2026-10-20 11:15:31 -07:00
shamoon 77a7fb40ec safe regex 2026-10-20 11:05:49 -07:00
15c95c251d Feature: OCR Templates (#13043)
[skip ci]

Signed-off-by: dependabot[bot] <support@github.com>
Co-Authored-By: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-Authored-By: stumpylog <797416+stumpylog@users.noreply.github.com>
Co-Authored-By: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-Authored-By: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-09 12:43:09 -07:00