Commit Graph
39 Commits
Author SHA1 Message Date
stumpylog c5fdba73a1 Fix: queue single document zone OCR runs instead of running them in the request
Rendering and OCR ran inside the HTTP request and could block other
requests for minutes. The action now validates, then queues the bulk
zone OCR operation for the selected file and returns its task id.
2026-11-19 15:25:48 -08:00
stumpylog a6b81b2d64 Fix: read zones before the consume transaction and write them before workflows
Zone OCR ran inside the consume transaction, read the working copy instead
of the archive, and ran after the document added workflows. Zones are now
read from the archive before the transaction and written just before
those workflows, keeping values supplied at consume time.
2026-11-19 15:09:41 -08:00
stumpylog 54be9dc9c5 Fix: finish manual and bulk zone OCR runs like other bulk edits
Runs reindexed each document separately, never sent document_updated
and did not bump modified for custom field writes. They now finish
through bulk_update_documents for the documents that got a value.
2026-11-19 14:18:30 -08:00
stumpylog 246f4e53b1 Chore: describe the zone OCR responses in the API schema
The run status and skip reason were string literals, and the test zone
and 404 bodies had no schema. Add enums and response serializers for
them. The JSON is unchanged.
2026-11-19 13:54:59 -08:00
stumpylog ff6b5f3820 Chore: dispatch zone OCR on the target, transform and data type enums
The string if/elif chains ended in catch-alls. Match on the enums with
assert_never instead. A stored value outside the choices now raises,
failing only that template.
2026-11-19 12:41:40 -08:00
stumpylog f35a2a4343 Chore: give zone OCR values a real type
Values were typed as object, and a result's value doubled as the
extracted text. Add a ZoneValue alias, carry the text separately, and
make the result dataclasses keyword-only.
2026-11-19 12:26:16 -08:00
stumpylog a490d9d348 Chore: type zone OCR results and reasons
Reasons become a StrEnum, the run response has a serializer instead of an
untyped dict list, and the zone preview returns its dataclass. The JSON is
unchanged.
2026-11-19 10:15:43 -08:00
stumpylog 0f4a71ed88 Fix: let documents without an owner use every OCR template
Unowned documents only matched unowned templates, so they never matched
templates created in the UI. Consider every template, as matching does.
2026-11-19 08:43:29 -08:00
stumpylog 0e7cbda3d9 Fix: don't pass raw text on when a zone transform fails
A date or numeric transform that found nothing returned the OCR text, so the
field got junk. Return nothing and report transform_failed, which also blocks
a combined field.
2026-11-19 08:19:07 -08:00
stumpylog fcc465ec2e Fix: reject over-long text zone values instead of truncating
A truncated URL is a different URL. Let field validation reject string and URL
values that are too long.
2026-11-19 08:10:35 -08:00
stumpylog 7b25551e1e Fix: drop only the ASN when a zone's ASN is already in use
A duplicate ASN failed the save and rolled back every write of the template.
Check first and report the zone as duplicate_asn instead.
2026-11-19 08:06:08 -08:00
stumpylog 3504ad07c6 Fix: keep zone number parsing linear and cap its input
Two patterns backtracked quadratically on long OCR text, taking seconds on a
large zone. Rewrite both, skip numbers and amounts longer than 256 characters,
and search only the first 256 characters for a date.
2026-11-19 08:02:51 -08:00
stumpylog 6fc4526947 Fix: keep the currency and validate monetary zone values
OCR dropped currency codes and stored amounts the API would reject. Keep an
ISO code or locale currency symbol, read parentheses as negative, and validate
with the same rule as the API.
2026-11-19 07:34:16 -08:00
stumpylog 218122452c Chore: test zone OCR outcomes instead of mocking internals
Run real pdftoppm and tesseract on a sample page, and stop patching
_process_template in the extraction pipeline tests.
2026-11-19 07:21:28 -08:00
stumpylog 409539d4d1 Chore: render zone OCR pages through the shared pdftoppm helper
pdftoppm was called in three places with different options. Missing-binary handling is dropped since pdftoppm and tesseract are required at startup.
2026-11-18 14:51:40 -08:00
stumpylog c1bb327f1f Fix: keep OCR values intact when combining zones into one field
Separator cleanup ran over the extracted values too, so a negative amount lost its sign. A zone failing validation also no longer lets a partial value through.
2026-11-18 14:20:11 -08:00
stumpylog 0e6d06c255 Feature: find dates after a label in OCR zones
A crop may include a label, such as "Datum: 13.04.2026", and those dates never parsed unless they were ISO.
2026-11-18 13:56:25 -08:00
stumpylog acf7ba346c Fix: parse OCR zone dates strictly using the zone's language
Dates ignored the document's day/month order, and words like "now" or a bare "5" were accepted as dates.
2026-11-18 13:46:21 -08:00
stumpylog 63a87dd11b Fix: parse OCR numbers with Babel using the zone's language
Separators were guessed from the text, so "12.50" became 1250 for integer fields and the thousands and decimal separators were often swapped.
2026-11-18 13:34:26 -08:00
stumpylog dc16714e19 Chore: align zone OCR test naming, URLs and docstrings 2026-11-18 13:03:41 -08:00
stumpylog 044da9a627 Chore: move misplaced zone OCR tests 2026-11-18 12:55:53 -08:00
stumpylog 093c18e68b Chore: parametrize near-duplicate zone OCR tests 2026-11-18 12:52:03 -08:00
stumpylog d706904a18 Chore: remove duplicate zone OCR tests and assert on untested ones 2026-11-18 12:48:20 -08:00
stumpylog e03faab730 Chore: share zone OCR mock fixtures and helpers 2026-11-18 12:41:46 -08:00
stumpylog 9ff48ecfce Chore: add CustomFieldFactory 2026-11-18 12:35:38 -08:00
stumpylog 65fdfcdea7 Chore: default OcrTemplateFactory to MATCH_ANY and drop redundant test arguments
Match the model's default matching, so tests stop restating it along with other factory defaults.
2026-11-18 12:33:49 -08:00
stumpylog 9d5c7abfaa Chore: convert zone OCR database tests to pytest
TestRunZoneExtraction used the real scratch dir, so it failed when run on its own.
2026-11-18 08:10:27 -08:00
stumpylog 1c685eb85a Chore: convert zone OCR extraction and render tests to pytest 2026-11-18 08:02:23 -08:00
stumpylog 44bbfd8932 Chore: convert zone OCR value function tests to pytest 2026-11-18 07:57:14 -08:00
stumpylog 4788ac602b Fix: limit manually run OCR templates to those the requester can view
A user editing someone else's document could run the owner's private
templates and see their names. Manual runs now use templates both can view.
2026-11-18 07:55:52 -08:00
shamoon e63f27a712 Crap missed versioning stuff, default to latest but allow specific 2026-11-09 09:19:30 -08:00
shamoonandOpenAI Codex 54f7f43f35 Reuse OCR page renders and simplify value conversion
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:07:00 -07:00
shamoonandOpenAI Codex 94da3d4c57 Report actual OCR extraction outcomes instead of existing field values
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:02:23 -07:00
shamoonandOpenAI Codex 453e89d6dd Prepare and atomically apply OCR template field updates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 17:00:03 -07:00
shamoonandOpenAI Codex 6562415894 Unify OCR zone extraction for preview and saved templates
Co-authored-by: OpenAI Codex <codex@openai.com>
2026-10-20 16:58:10 -07:00
shamoon bedefd85f2 Remove the handlers, make the matching work 2026-10-20 13:39:09 -07:00
shamoon e976b2ec7a Tests 2026-10-20 11:23:44 -07:00
shamoon 9783c4995f Fix the tests for now 2026-10-20 10:42:12 -07:00
15c95c251d Feature: OCR Templates (#13043)
[skip ci]

Signed-off-by: dependabot[bot] <support@github.com>
Co-Authored-By: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-Authored-By: stumpylog <797416+stumpylog@users.noreply.github.com>
Co-Authored-By: GitHub Actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-Authored-By: shamoon <4887959+shamoon@users.noreply.github.com>
2026-09-09 12:43:09 -07:00