paperless-ngx/docs/superpowers/plans/2026-05-19-workflow-runner-refactor.md

Workflow Runner Refactor Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Updated 2026-09-24: line references below are refreshed against current dev. More importantly: the motivating bugs (#12386 and the tag/m2m race) are already fixed by #12389 and #13178, independently of this plan — see "Status update (2026-09-24)" in the linked spec. This is now a structural cleanup (kill use_overrides dual-mode branching and the original_file parameter plumbing), not an active-bug fix. Task 7 is rescoped accordingly (the regression test it originally asked for already exists).

Goal: Replace the use_overrides dual-mode branching in run_workflows with a polymorphic WorkflowRunContext, and make the workflow-execution → file-rename sequence deterministic via a ContextVar guard.

Architecture: A new documents/workflows/context.py module defines a WorkflowRunContext Protocol and two implementations — ConsumptionContext (wraps ConsumableDocument + DocumentMetadataOverrides) and PersistedContext (wraps a real Document). run_workflows becomes a flat match-and-dispatch loop with no mode flag. A module-level ContextVar guard suppresses update_filename_and_move_files for the duration of a workflow run; the file rename is invoked once, explicitly, after the run.

Tech Stack: Python 3.12+, Django, Celery, pytest (pytest-xdist), uv, ruff.

Reference spec: docs/superpowers/specs/2026-05-19-workflow-runner-refactor-design.md

---

Conventions for every task

• All Python commands run from src/. Tests: uv run pytest. Lint: ruff check --fix then ruff format (global ruff, not uv run ruff).
• New test code: group tests under classes, put @pytest.mark.django_db per class where DB is needed, fully type-annotate every fixture parameter, fixture return type, and test signature.
• Commit after each task with the message shown in its final step.

File structure

File	Responsibility	Change
src/documents/workflows/context.py	ContextVar guard, WorkflowRunContext Protocol, ConsumptionContext, PersistedContext, build_workflow_context factory	Create
src/documents/signals/handlers.py	run_workflows rewritten as flat loop; update_filename_and_move_files split into receiver + move_files_for_document	Modify
src/documents/workflows/actions.py	Drop build_workflow_action_context; execute_email_action / execute_webhook_action take source_file instead of original_file	Modify
src/documents/tests/test_workflow_context.py	Unit tests for the guard and the two contexts	Create
src/documents/tests/test_workflows.py	Existing regression suite (must stay green) + new guard/race regression tests	Modify (add tests only)

---

Task 1: ContextVar guard

Files:

• Create: src/documents/workflows/context.py

• Test: src/documents/tests/test_workflow_context.py

• Step 1: Write the failing test

Create src/documents/tests/test_workflow_context.py:

from documents.workflows.context import workflow_guard
from documents.workflows.context import workflow_run_in_progress


class TestWorkflowGuard:
    def test_guard_not_set_by_default(self) -> None:
        assert workflow_run_in_progress() is False

    def test_guard_set_inside_context_manager(self) -> None:
        with workflow_guard():
            assert workflow_run_in_progress() is True
        assert workflow_run_in_progress() is False

    def test_guard_is_reentrant(self) -> None:
        with workflow_guard():
            with workflow_guard():
                assert workflow_run_in_progress() is True
            assert workflow_run_in_progress() is True
        assert workflow_run_in_progress() is False

    def test_guard_resets_on_exception(self) -> None:
        try:
            with workflow_guard():
                raise RuntimeError("boom")
        except RuntimeError:
            pass
        assert workflow_run_in_progress() is False

• Step 2: Run test to verify it fails

Run: uv run pytest documents/tests/test_workflow_context.py -v Expected: FAIL with ModuleNotFoundError: No module named 'documents.workflows.context'

• Step 3: Create the module with the guard

Create src/documents/workflows/context.py with only the guard for now:

import logging
from collections.abc import Iterator
from contextlib import contextmanager
from contextvars import ContextVar

logger = logging.getLogger("paperless.workflows")

_workflow_in_progress: ContextVar[bool] = ContextVar(
    "workflow_in_progress",
    default=False,
)


def workflow_run_in_progress() -> bool:
    """
    True while run_workflows is executing on the current thread/context.

    update_filename_and_move_files checks this and early-returns, so the file
    rename does not race the workflow's own metadata mutations.
    """
    return _workflow_in_progress.get()


@contextmanager
def workflow_guard() -> Iterator[None]:
    """
    Suppress update_filename_and_move_files for the duration of a workflow run.

    Token-based reset keeps this reentrancy-safe: nested workflow_guard() blocks
    (e.g. a workflow whose action triggers another save) restore the previous
    value rather than unconditionally clearing the flag.
    """
    token = _workflow_in_progress.set(True)
    try:
        yield
    finally:
        _workflow_in_progress.reset(token)

• Step 4: Run test to verify it passes

Run: uv run pytest documents/tests/test_workflow_context.py -v Expected: PASS (4 tests)

• Step 5: Lint and commit

ruff check --fix src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
ruff format src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git add src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git commit -m "Add ContextVar guard for workflow runner"

---

Task 2: WorkflowRunContext Protocol + ConsumptionContext

Files:

• Modify: src/documents/workflows/context.py
• Test: src/documents/tests/test_workflow_context.py

The ConsumptionContext absorbs the use_overrides=True branches currently in run_workflows (handlers.py:862-1051) and build_workflow_action_context (actions.py:33,53-83).

• Step 1: Write the failing test

Append to src/documents/tests/test_workflow_context.py:

import pytest

from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from documents.data_models import DocumentSource
from documents.models import Correspondent
from documents.models import WorkflowTrigger
from documents.workflows.context import ConsumptionContext


@pytest.mark.django_db
class TestConsumptionContext:
    @pytest.fixture
    def consumable(self, tmp_path) -> ConsumableDocument:
        staged = tmp_path / "staged.pdf"
        staged.write_bytes(b"%PDF-1.4")
        return ConsumableDocument(
            source=DocumentSource.ConsumeFolder,
            original_file=staged,
        )

    def test_source_and_staged_file_are_the_staged_path(
        self,
        consumable: ConsumableDocument,
    ) -> None:
        overrides = DocumentMetadataOverrides()
        ctx = ConsumptionContext(
            WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
            consumable,
            overrides,
        )
        assert ctx.source_file == consumable.original_file
        assert ctx.staged_file == consumable.original_file
        assert ctx.target is consumable

    def test_assignment_mutates_overrides_not_db(
        self,
        consumable: ConsumableDocument,
    ) -> None:
        correspondent = Correspondent.objects.create(name="ACME")
        overrides = DocumentMetadataOverrides()
        ctx = ConsumptionContext(
            WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
            consumable,
            overrides,
        )

        class FakeAction:
            assign_correspondent = correspondent
            assign_document_type = None
            assign_storage_path = None
            assign_owner = None
            assign_title = None
            has_assign_tags = False
            has_assign_view_users = False
            has_assign_view_groups = False
            has_assign_change_users = False
            has_assign_change_groups = False
            has_assign_custom_fields = False

        ctx.apply_assignment(FakeAction(), logging_group=None)
        assert overrides.correspondent_id == correspondent.pk

    def test_log_action_collects_messages(
        self,
        consumable: ConsumableDocument,
    ) -> None:
        ctx = ConsumptionContext(
            WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
            consumable,
            DocumentMetadataOverrides(),
        )
        ctx.log_action("hello", logging_group=None)
        assert ctx.messages == ["hello"]

• Step 2: Run test to verify it fails

Run: uv run pytest documents/tests/test_workflow_context.py::TestConsumptionContext -v Expected: FAIL with ImportError: cannot import name 'ConsumptionContext'

• Step 3: Add the Protocol and ConsumptionContext

Add to src/documents/workflows/context.py (after the guard). Add these imports at the top of the file:

import uuid
from pathlib import Path
from typing import Protocol

from django.contrib.auth.models import User
from django.utils import timezone

from documents.data_models import ConsumableDocument
from documents.data_models import DocumentMetadataOverrides
from documents.models import Correspondent
from documents.models import Document
from documents.models import DocumentType
from documents.models import Workflow
from documents.models import WorkflowAction
from documents.models import WorkflowRun
from documents.models import WorkflowTrigger
from documents.workflows.mutations import apply_assignment_to_overrides
from documents.workflows.mutations import apply_removal_to_overrides

Then append:

class WorkflowRunContext(Protocol):
    """
    Uniform surface a workflow action operates on, independent of whether the
    run targets a freshly-consumed document (overrides) or a persisted one.
    """

    trigger_type: WorkflowTrigger.WorkflowTriggerType

    @property
    def target(self) -> Document | ConsumableDocument:
        """The object passed to matching and to email/webhook/trash actions."""
        ...

    @property
    def source_file(self) -> Path:
        """Best-effort current on-disk location of the document file."""
        ...

    @property
    def staged_file(self) -> Path | None:
        """
        The staged (pre-final-move) path, when the file is not yet at its
        final storage location; otherwise None. Used by password removal.
        """
        ...

    def refresh(self, logging_group: uuid.UUID | None) -> bool:
        """Reload state before matching. Return False to stop the run."""
        ...

    def build_placeholder_context(self) -> dict:
        """Context dict for email/webhook placeholder parsing."""
        ...

    def apply_assignment(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None: ...

    def apply_removal(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None: ...

    def log_action(self, message: str, logging_group: uuid.UUID | None) -> None:
        """Record an 'applying action' message."""
        ...

    def persist(self) -> None:
        """Commit accumulated mutations (no-op for the consumption flow)."""
        ...

    def record_run(self, workflow: Workflow) -> None:
        """Create the WorkflowRun audit row."""
        ...

    def finalize_file_location(self) -> None:
        """Run the one-time file rename after the workflow run completes."""
        ...


class ConsumptionContext:
    """Workflow run against a document still being consumed."""

    def __init__(
        self,
        trigger_type: WorkflowTrigger.WorkflowTriggerType,
        consumable: ConsumableDocument,
        overrides: DocumentMetadataOverrides,
    ) -> None:
        self.trigger_type = trigger_type
        self.consumable = consumable
        self.overrides = overrides
        self.messages: list[str] = []

    @property
    def target(self) -> Document | ConsumableDocument:
        return self.consumable

    @property
    def source_file(self) -> Path:
        return self.consumable.original_file

    @property
    def staged_file(self) -> Path | None:
        return self.consumable.original_file

    def refresh(self, logging_group: uuid.UUID | None) -> bool:
        return True

    def build_placeholder_context(self) -> dict:
        overrides = self.overrides
        correspondent_obj = (
            Correspondent.objects.filter(pk=overrides.correspondent_id).first()
            if overrides.correspondent_id
            else None
        )
        document_type_obj = (
            DocumentType.objects.filter(pk=overrides.document_type_id).first()
            if overrides.document_type_id
            else None
        )
        owner_obj = (
            User.objects.filter(pk=overrides.owner_id).first()
            if overrides.owner_id
            else None
        )
        filename = (
            self.consumable.original_file if self.consumable.original_file else ""
        )
        return {
            "title": overrides.title
            if overrides.title
            else str(self.consumable.original_file),
            "doc_url": "",
            "correspondent": correspondent_obj.name if correspondent_obj else "",
            "document_type": document_type_obj.name if document_type_obj else "",
            "owner_username": owner_obj.username if owner_obj else "",
            "filename": filename,
            "current_filename": filename,
            "added": timezone.localtime(timezone.now()),
            "created": overrides.created,
            "id": "",
        }

    def apply_assignment(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None:
        apply_assignment_to_overrides(action, self.overrides)

    def apply_removal(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None:
        apply_removal_to_overrides(action, self.overrides)

    def log_action(self, message: str, logging_group: uuid.UUID | None) -> None:
        self.messages.append(message)

    def persist(self) -> None:
        return None

    def record_run(self, workflow: Workflow) -> None:
        WorkflowRun.objects.create(
            workflow=workflow,
            type=self.trigger_type,
            document=None,
        )

    def finalize_file_location(self) -> None:
        return None

Note: the build_placeholder_context body is moved verbatim from the use_overrides branch of build_workflow_action_context (actions.py:53-83).

• Step 4: Run test to verify it passes

Run: uv run pytest documents/tests/test_workflow_context.py::TestConsumptionContext -v Expected: PASS (3 tests)

• Step 5: Lint and commit

ruff check --fix src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
ruff format src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git add src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git commit -m "Add WorkflowRunContext protocol and ConsumptionContext"

---

Task 3: PersistedContext + build_workflow_context factory

Files:

• Modify: src/documents/workflows/context.py
• Test: src/documents/tests/test_workflow_context.py

PersistedContext absorbs the use_overrides=False branches of run_workflows (handlers.py:904-1046) and build_workflow_action_context (actions.py:35-51).

• Step 1: Write the failing test

Append to src/documents/tests/test_workflow_context.py:

from documents.workflows.context import PersistedContext
from documents.workflows.context import build_workflow_context


@pytest.mark.django_db
class TestPersistedContext:
    @pytest.fixture
    def document(self) -> Document:
        return Document.objects.create(
            title="doc",
            mime_type="application/pdf",
            checksum="abc123",
        )

    def test_source_file_is_document_source_path_without_staged(
        self,
        document: Document,
    ) -> None:
        ctx = PersistedContext(
            WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
            document,
            staged_file=None,
        )
        assert ctx.source_file == document.source_path
        assert ctx.staged_file is None
        assert ctx.target is document

    def test_source_file_prefers_staged_path(
        self,
        document: Document,
        tmp_path,
    ) -> None:
        staged = tmp_path / "staged.pdf"
        staged.write_bytes(b"%PDF-1.4")
        ctx = PersistedContext(
            WorkflowTrigger.WorkflowTriggerType.DOCUMENT_ADDED,
            document,
            staged_file=staged,
        )
        assert ctx.source_file == staged
        assert ctx.staged_file == staged

    def test_refresh_returns_false_when_document_deleted(
        self,
        document: Document,
    ) -> None:
        ctx = PersistedContext(
            WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
            document,
            staged_file=None,
        )
        Document.objects.filter(pk=document.pk).delete()
        assert ctx.refresh(logging_group=None) is False


@pytest.mark.django_db
class TestBuildWorkflowContext:
    def test_overrides_present_builds_consumption_context(self, tmp_path) -> None:
        from documents.data_models import DocumentSource

        consumable = ConsumableDocument(
            source=DocumentSource.ConsumeFolder,
            original_file=tmp_path / "f.pdf",
        )
        ctx = build_workflow_context(
            trigger_type=WorkflowTrigger.WorkflowTriggerType.CONSUMPTION,
            document=consumable,
            overrides=DocumentMetadataOverrides(),
            staged_file=None,
        )
        assert isinstance(ctx, ConsumptionContext)

    def test_no_overrides_builds_persisted_context(self) -> None:
        document = Document.objects.create(
            title="d",
            mime_type="application/pdf",
            checksum="zzz",
        )
        ctx = build_workflow_context(
            trigger_type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
            document=document,
            overrides=None,
            staged_file=None,
        )
        assert isinstance(ctx, PersistedContext)

• Step 2: Run test to verify it fails

Run: uv run pytest documents/tests/test_workflow_context.py::TestPersistedContext -v Expected: FAIL with ImportError: cannot import name 'PersistedContext'

• Step 3: Add PersistedContext and the factory

Add these imports to src/documents/workflows/context.py:

from documents.workflows.mutations import apply_assignment_to_document
from documents.workflows.mutations import apply_removal_to_document

Append to src/documents/workflows/context.py:

# Fields a workflow ASSIGNMENT/REMOVAL action can set directly on a Document.
# Deliberately excludes filename / archive_filename: those are managed only by
# move_files_for_document. A workflow never sets them, and writing a stale
# in-memory value here would revert a concurrent (cross-process) file move.
# modified has auto_now=True but is not auto-added when update_fields is given.
_WORKFLOW_SAVE_FIELDS = [
    "title",
    "correspondent",
    "document_type",
    "storage_path",
    "owner",
    "modified",
]


class PersistedContext:
    """Workflow run against a document already in the database."""

    def __init__(
        self,
        trigger_type: WorkflowTrigger.WorkflowTriggerType,
        document: Document,
        staged_file: Path | None,
    ) -> None:
        self.trigger_type = trigger_type
        self.document = document
        self._staged_file = staged_file

    @property
    def target(self) -> Document | ConsumableDocument:
        return self.document

    @property
    def source_file(self) -> Path:
        return self._staged_file if self._staged_file else self.document.source_path

    @property
    def staged_file(self) -> Path | None:
        return self._staged_file

    def refresh(self, logging_group: uuid.UUID | None) -> bool:
        try:
            # run_workflows may be called repeatedly from bulk_update_documents;
            # refresh so matching data is current and we do not overwrite work
            # done by another process.
            self.document.refresh_from_db()
        except Document.DoesNotExist:
            logger.info(
                "Document no longer exists, skipping remaining workflows",
                extra={"group": logging_group},
            )
            return False
        if self.document.is_deleted:
            logger.info(
                "Document was moved to trash, skipping remaining workflows",
                extra={"group": logging_group},
            )
            return False
        return True

    def build_placeholder_context(self) -> dict:
        from django.conf import settings

        document = self.document
        return {
            "title": document.title,
            "doc_url": f"{settings.PAPERLESS_URL}{settings.BASE_URL}documents/{document.pk}/",
            "correspondent": document.correspondent.name
            if document.correspondent
            else "",
            "document_type": document.document_type.name
            if document.document_type
            else "",
            "owner_username": document.owner.username if document.owner else "",
            "filename": document.original_filename or "",
            "current_filename": document.filename or "",
            "added": timezone.localtime(document.added),
            "created": document.created,
            "id": document.pk,
        }

    def apply_assignment(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None:
        apply_assignment_to_document(action, self.document, logging_group)

    def apply_removal(
        self,
        action: WorkflowAction,
        logging_group: uuid.UUID | None,
    ) -> None:
        apply_removal_to_document(action, self.document)

    def log_action(self, message: str, logging_group: uuid.UUID | None) -> None:
        logger.info(message, extra={"group": logging_group})

    def persist(self) -> None:
        # limit title to 128 characters
        self.document.title = self.document.title[:128]
        self.document.save(update_fields=_WORKFLOW_SAVE_FIELDS)

    def record_run(self, workflow: Workflow) -> None:
        WorkflowRun.objects.create(
            workflow=workflow,
            type=self.trigger_type,
            document=self.document,
        )

    def finalize_file_location(self) -> None:
        # Imported here to avoid a circular import (handlers imports this module).
        from documents.signals.handlers import move_files_for_document

        try:
            self.document.refresh_from_db()
        except Document.DoesNotExist:
            return
        if self.document.is_deleted:
            return
        move_files_for_document(self.document)


def build_workflow_context(
    *,
    trigger_type: WorkflowTrigger.WorkflowTriggerType,
    document: Document | ConsumableDocument,
    overrides: DocumentMetadataOverrides | None,
    staged_file: Path | None,
) -> WorkflowRunContext:
    """
    Pick the context implementation from the call shape: a non-None `overrides`
    means the consumption flow (ConsumableDocument); otherwise a persisted
    Document.
    """
    if overrides is not None:
        assert isinstance(document, ConsumableDocument)
        return ConsumptionContext(trigger_type, document, overrides)
    assert isinstance(document, Document)
    return PersistedContext(trigger_type, document, staged_file)

Note: build_placeholder_context here is moved verbatim from the non-use_overrides branch of build_workflow_action_context (actions.py:35-51). _WORKFLOW_SAVE_FIELDS and persist() reproduce the save at handlers.py:1028-1037 — keep the field list and the comment.

• Step 4: Run test to verify it passes

Run: uv run pytest documents/tests/test_workflow_context.py -v Expected: PASS (all tests, ~13)

• Step 5: Lint and commit

ruff check --fix src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
ruff format src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git add src/documents/workflows/context.py src/documents/tests/test_workflow_context.py
git commit -m "Add PersistedContext and build_workflow_context factory"

---

Task 4: Extract move_files_for_document from the rename receiver

Files:

• Modify: src/documents/signals/handlers.py:437-673

This task is a pure extraction — no behavior change yet. The guard check is added in Task 6.

Note: re-verify these line numbers against current handlers.py before starting — this file also received a checksum-based "already moved" recovery fix (#12389, _path_matches_checksum at handlers.py:416-420 plus changes inside validate_move) since this plan was written. That logic is unrelated to this extraction and must be carried over unchanged inside the extracted body — do not simplify or remove it.

• Step 1: Confirm the regression baseline is green

Run: uv run pytest documents/tests/test_file_handling.py -q Expected: PASS (records the pre-change baseline for the rename logic)

• Step 2: Split the function

In src/documents/signals/handlers.py, replace the update_filename_and_move_files definition (currently def update_filename_and_move_files(...) at line 440, through the end of its body at line 673) with two definitions:

@receiver(models.signals.post_save, sender=CustomFieldInstance, weak=False)
@receiver(models.signals.m2m_changed, sender=Document.tags.through, weak=False)
@receiver(models.signals.post_save, sender=Document, weak=False)
def update_filename_and_move_files(
    sender,
    instance: Document | CustomFieldInstance,
    **kwargs,
) -> None:
    if isinstance(instance, CustomFieldInstance):
        if not _filename_template_uses_custom_fields(instance.document):
            return
        instance = instance.document

    move_files_for_document(instance)


def move_files_for_document(instance: Document) -> None:
    def validate_move(
        instance, old_path: Path, new_path: Path, root: Path
    ) -> None: ...  # body unchanged from current lines 444-463

Move the entire body currently between line 450 (def validate_move) and line 673 into move_files_for_document, unchanged, with one edit: the recursive call at the end (currently handlers.py:665-673, which today calls the whole receiver directly — update_filename_and_move_files(Document, version_doc), passing a synthetic Document sender to satisfy the receiver's signature) must instead call the new function directly:

    # Keep version files in sync with root
    if instance.root_document_id is None:
        for version_doc in Document.objects.filter(root_document_id=instance.pk).only(
            "pk",
        ):
            move_files_for_document(version_doc)

• Step 3: Run the rename tests

Run: uv run pytest documents/tests/test_file_handling.py -q Expected: PASS (identical to Step 1 — extraction is behavior-preserving)

• Step 4: Run the workflow suite

Run: uv run pytest documents/tests/test_workflows.py -q Expected: PASS (unchanged)

• Step 5: Lint and commit

ruff check --fix src/documents/signals/handlers.py
ruff format src/documents/signals/handlers.py
git add src/documents/signals/handlers.py
git commit -m "Extract move_files_for_document from rename receiver"

---

Task 5: Rewrite run_workflows to use contexts

Files:

• Modify: src/documents/signals/handlers.py (run_workflows, run_workflows_added)
• Modify: src/documents/workflows/actions.py (execute_email_action, execute_webhook_action, delete build_workflow_action_context)

This removes the use_overrides flag, the caller_supplied_original_file flag, and the original_file threading into execute_* helpers. It does not yet add the guard (Task 6).

• Step 1: Confirm baseline

Run: uv run pytest documents/tests/test_workflows.py -q Expected: PASS (baseline before the rewrite)

• Step 2: Update actions.py — execute_email_action signature

In src/documents/workflows/actions.py, change execute_email_action so the parameter original_file: Path is renamed to source_file: Path, and replace every use of original_file in its body (actions.py:158-167) with source_file. The logic is otherwise unchanged.

• Step 3: Update actions.py — execute_webhook_action signature

In execute_webhook_action, rename the parameter original_file: Path to source_file: Path and replace the two uses (actions.py:245,250) with source_file.

• Step 4: Delete build_workflow_action_context

Delete the entire build_workflow_action_context function from src/documents/workflows/actions.py (actions.py:26-83) — its two branches now live in ConsumptionContext.build_placeholder_context and PersistedContext.build_placeholder_context.

• Step 5: Rewrite run_workflows in handlers.py

In src/documents/signals/handlers.py, add to the imports:

from documents.workflows.context import WorkflowRunContext
from documents.workflows.context import build_workflow_context

and remove the now-unused build_workflow_action_context import.

Replace the entire run_workflows function (handlers.py:862-1051) with:

def run_workflows(
    trigger_type: WorkflowTrigger.WorkflowTriggerType,
    document: Document | ConsumableDocument,
    workflow_to_run: Workflow | None = None,
    logging_group: uuid.UUID | None = None,
    overrides: DocumentMetadataOverrides | None = None,
    original_file: Path | None = None,
) -> tuple[DocumentMetadataOverrides, str] | None:
    """
    Execute workflows matching a document for the given trigger.

    For the consumption flow (`overrides` provided) actions mutate the overrides
    and the function returns `(overrides, messages)`. Otherwise actions mutate
    the persisted Document and the function returns None.

    `original_file`, when given, is the staged path of a freshly-consumed file
    that has not yet been moved into its final storage location.
    """
    if isinstance(document, Document) and document.root_document_id is not None:
        logger.debug(
            "Skipping workflow execution for version document %s",
            document.pk,
        )
        return None

    context: WorkflowRunContext = build_workflow_context(
        trigger_type=trigger_type,
        document=document,
        overrides=overrides,
        staged_file=original_file,
    )

    for workflow in get_workflows_for_trigger(trigger_type, workflow_to_run):
        if not context.refresh(logging_group):
            break

        if not matching.document_matches_workflow(
            context.target,
            workflow,
            trigger_type,
        ):
            continue

        action: WorkflowAction
        has_move_to_trash_action = False
        for action in workflow.actions.order_by("order", "pk"):
            context.log_action(f"Applying {action} from {workflow}", logging_group)

            if action.type == WorkflowAction.WorkflowActionType.ASSIGNMENT:
                context.apply_assignment(action, logging_group)
            elif action.type == WorkflowAction.WorkflowActionType.REMOVAL:
                context.apply_removal(action, logging_group)
            elif action.type == WorkflowAction.WorkflowActionType.EMAIL:
                execute_email_action(
                    action,
                    context.target,
                    context.build_placeholder_context(),
                    logging_group,
                    context.source_file,
                    trigger_type,
                )
            elif action.type == WorkflowAction.WorkflowActionType.WEBHOOK:
                execute_webhook_action(
                    action,
                    context.target,
                    context.build_placeholder_context(),
                    logging_group,
                    context.source_file,
                )
            elif action.type == WorkflowAction.WorkflowActionType.PASSWORD_REMOVAL:
                execute_password_removal_action(
                    action,
                    context.target,
                    logging_group,
                    source_file=context.staged_file,
                )
            elif action.type == WorkflowAction.WorkflowActionType.MOVE_TO_TRASH:
                has_move_to_trash_action = True

        context.persist()
        context.record_run(workflow)

        if has_move_to_trash_action:
            execute_move_to_trash_action(action, context.target, logging_group)

    if isinstance(context, ConsumptionContext):
        return context.overrides, "\n".join(context.messages)
    return None

Add the import from documents.workflows.context import ConsumptionContext to handlers.py (used for the isinstance at the end).

Note on staged_file for password removal: previously the action received original_file if caller_supplied_original_file else None. PersistedContext. staged_file is exactly the constructor staged_file arg, which is original_file — so DOCUMENT_ADDED gets the staged path and DOCUMENT_UPDATED / SCHEDULED get None, preserving the old behavior. For the consumption flow, execute_password_removal_action takes the ConsumableDocument branch and ignores source_file entirely.

• Step 6: Update run_workflows_added

run_workflows_added (handlers.py:811-824) already forwards original_file to run_workflows. Leave it as-is — it still receives original_file from the document_consumption_finished signal and passes it through; run_workflows now routes it into PersistedContext construction.

• Step 7: Run the full workflow suite

Run: uv run pytest documents/tests/test_workflows.py -q Expected: PASS — identical pass count to Step 1. If any test needs editing, STOP: that is a behavior change, not a refactor outcome — investigate before continuing.

• Step 8: Run consumer + matching suites

Run: uv run pytest documents/tests/test_consumer.py documents/tests/test_matchables.py -q Expected: PASS

• Step 9: Lint and commit

ruff check --fix src/documents/signals/handlers.py src/documents/workflows/actions.py
ruff format src/documents/signals/handlers.py src/documents/workflows/actions.py
git add src/documents/signals/handlers.py src/documents/workflows/actions.py
git commit -m "Refactor run_workflows around WorkflowRunContext, drop use_overrides"

---

Task 6: Wire the guard into run_workflows and the rename receiver

Files:

• Modify: src/documents/signals/handlers.py (run_workflows, update_filename_and_move_files, PersistedContext use via finalize_file_location)

• Test: src/documents/tests/test_workflows.py

• Step 1: Write the failing test

Append a new test class to src/documents/tests/test_workflows.py. The action must assign both a tag and a correspondent/storage-path (not correspondent alone) — a correspondent-only assignment never fires m2m_changed mid-run, so it would call move_files_for_document exactly once even without the guard, and the test would pass vacuously. Use the same shape as the existing test_document_updated_workflow_assignment_storage_path_persists_with_tag_assignment (test_workflows.py:3030) for the Document/StoragePath/Workflow/ASSIGNMENT setup, and call run_workflows(...) directly as that test does. The new test:

class TestWorkflowRenameSequencing(TestCase):
    @pytest.mark.django_db
    def test_rename_suppressed_during_run_then_runs_once(self) -> None:
        """
        update_filename_and_move_files must not run while a workflow is
        executing; the move runs exactly once afterwards.
        """
        from unittest import mock

        from documents.signals import handlers

        # ... set up a Document, a StoragePath whose template depends on
        # correspondent, and a DOCUMENT_UPDATED workflow with one ASSIGNMENT
        # action assigning BOTH a tag and that correspondent (reuse the setup
        # from test_document_updated_workflow_assignment_storage_path_persists_with_tag_assignment) ...

        with mock.patch.object(
            handlers,
            "move_files_for_document",
            wraps=handlers.move_files_for_document,
        ) as move_spy:
            run_workflows(
                trigger_type=WorkflowTrigger.WorkflowTriggerType.DOCUMENT_UPDATED,
                document=document,
            )

        # The m2m_changed / post_save receiver must NOT have driven the move
        # mid-run; only the single explicit finalize call remains.
        assert move_spy.call_count == 1

The implementer should flesh out the document/workflow setup using the patterns already present in test_workflows.py. The assertion that matters: move_files_for_document is called exactly once, via the explicit finalize_file_location(), not once-per-save() from signals. Pre-guard, current code (post-#13178) calls it twice for this setup: once from the m2m_changed receiver when apply_assignment_to_document fetches a fresh instance to add the tag (a self-consistent no-op move against still-current DB state), and once from the final document.save()'s post_save. So this test is a genuine (if narrower) regression check, not a vacuous one — it's just checking call count / ordering discipline rather than a wrong final path, since #13178 already ensures the final path is correct either way.

• Step 2: Run test to verify it fails

Run: uv run pytest documents/tests/test_workflows.py::TestWorkflowRenameSequencing -v Expected: FAIL — without the guard, move_files_for_document is invoked multiple times (via post_save / m2m_changed), so call_count != 1.

• Step 3: Add the guard check to the receiver

In src/documents/signals/handlers.py, add the early-return to the update_filename_and_move_files receiver. Add the import:

from documents.workflows.context import workflow_run_in_progress

and make the receiver:

def update_filename_and_move_files(
    sender,
    instance: Document | CustomFieldInstance,
    **kwargs,
) -> None:
    if workflow_run_in_progress():
        # A workflow run is mutating this document; the rename is deferred to a
        # single explicit call in run_workflows after the run completes.
        return

    if isinstance(instance, CustomFieldInstance):
        if not _filename_template_uses_custom_fields(instance.document):
            return
        instance = instance.document

    move_files_for_document(instance)

• Step 4: Wrap the run and add the explicit rename in run_workflows

In run_workflows, add the import:

from documents.workflows.context import workflow_guard

Wrap the for workflow in ... loop in with workflow_guard():, and call context.finalize_file_location() after the loop (outside the with block):

    with workflow_guard():
        for workflow in get_workflows_for_trigger(trigger_type, workflow_to_run):
            ...  # loop body unchanged
            if has_move_to_trash_action:
                execute_move_to_trash_action(action, context.target, logging_group)

    context.finalize_file_location()

    if isinstance(context, ConsumptionContext):
        return context.overrides, "\n".join(context.messages)
    return None

ConsumptionContext.finalize_file_location() is a no-op; PersistedContext's runs move_files_for_document once against the refreshed, non-deleted document.

• Step 5: Run the new test

Run: uv run pytest documents/tests/test_workflows.py::TestWorkflowRenameSequencing -v Expected: PASS

• Step 6: Run the full workflow + file-handling suites

Run: uv run pytest documents/tests/test_workflows.py documents/tests/test_file_handling.py -q Expected: PASS — no pre-existing test changed.

• Step 7: Lint and commit

ruff check --fix src/documents/signals/handlers.py src/documents/tests/test_workflows.py
ruff format src/documents/signals/handlers.py src/documents/tests/test_workflows.py
git add src/documents/signals/handlers.py src/documents/tests/test_workflows.py
git commit -m "Defer workflow file rename via ContextVar guard"

---

Task 7: Confirm pre-existing metadata-vs-rename regression coverage still holds

Files: none (verification only — no new test)

Rescoped 2026-09-24: this task originally asked for a new regression test for the #12386 metadata-vs-rename race. That race was independently fixed by #13178 (merged 2026-07-20, after this plan was written), whose only added test is test_document_updated_workflow_assignment_storage_path_persists_with_tag_assignment in test_workflows.py. Writing another test asserting the same final-state outcome would duplicate that coverage. Task 6's TestWorkflowRenameSequencing already adds the piece that test doesn't cover — call-count/ordering discipline on move_files_for_document — so this task is now pure verification that nothing regressed.

• Step 1: Run the pre-existing regression tests by name

Run (all three are in the TestWorkflows class, test_workflows.py:78-5123):

uv run pytest \
  "documents/tests/test_workflows.py::TestWorkflows::test_document_updated_workflow_assignment_storage_path_persists_with_tag_assignment" \
  "documents/tests/test_workflows.py::TestWorkflows::test_document_updated_workflow_assignment_persists_when_removing_trigger_tag" \
  "documents/tests/test_workflows.py::TestWorkflows::test_workflow_document_updated_does_not_overwrite_filename" \
  -v

(Confirm the exact enclosing class name for each with grep -n "class Test\|def test_" src/documents/tests/test_workflows.py before running — class boundaries may have shifted since this plan was written.)

Expected: PASS, unchanged — same assertions, no edits needed. If any of these needs modification to pass after the refactor, that is a behavior regression introduced by this plan, not an acceptable side effect — stop and investigate before continuing.

• Step 2: No commit

This task makes no code changes. If Step 1 required a fix elsewhere, that fix belongs to whichever earlier task's commit introduced the regression — amend that task's changes (as a new commit, not --amend) rather than committing here.

---

Task 8: Full verification

Files: none (verification only)

• Step 1: Run the full backend suite

Run: uv run pytest -m "not live" -q Expected: PASS. Investigate any failure before claiming completion.

• Step 2: Lint the whole change

Run: ruff check src/documents/ then ruff format --check src/documents/ Expected: clean.

• Step 3: Type-check

Run: uv run mypy documents/workflows/context.py documents/signals/handlers.py documents/workflows/actions.py Expected: no new errors beyond .mypy-baseline.txt. Per CLAUDE.md, do not attempt to clear the baseline — only ensure you introduced nothing new.

• Step 4: Confirm no stray references

Run: git grep -n "build_workflow_action_context\|caller_supplied_original_file\|use_overrides" src/ Expected: no matches (all three are fully removed).

• Step 5: Final commit (if Step 2-4 required fixes)

git add -A
git commit -m "Clean up after workflow runner refactor"

---

Self-review notes (already applied)

• Spec coverage: §1 Protocol → Tasks 2-3; §2 branch-free run_workflows → Task 5; §3 source_file / staged-path relocation → Tasks 3, 5; §4 guard + sequencing → Tasks 4, 6; deferred password-removal hook left as-is (Task 5 note); testing → Tasks 1-3, 6, 7 (rescoped), 8.
• update_fields exclusion kept (_WORKFLOW_SAVE_FIELDS in Task 3) — per the design decision that it guards a cross-process hazard the guard does not cover. No task removes it.
• Type consistency: WorkflowRunContext surface (target, source_file, staged_file, refresh, build_placeholder_context, apply_assignment, apply_removal, log_action, persist, record_run, finalize_file_location) is identical across the Protocol, both implementations, and all call sites in the rewritten run_workflows.

2026-09-24 status update

• All line-number references throughout this plan were re-verified against current dev and corrected (handlers.py shifted by ~6-8 lines throughout due to unrelated intervening changes, chiefly #12389's _path_matches_checksum addition).
• The two concrete bugs motivating this refactor are already fixed independently, by different mechanisms, without this plan: #12389 (cross-process already-moved-file race, checksum-based recovery in validate_move) and #13178 (intra-workflow tag/m2m_changed clobbering unsaved fields, fixed by fetching a fresh Document instance for tag mutation — see mutations.py:26-31). Neither fix uses WorkflowRunContext or the ContextVar guard.
• Task 4 now calls out explicitly that the checksum-recovery logic inside validate_move must be carried over unchanged during extraction, and that the version-document recursive call site (handlers.py:670-673) needs updating to call the extracted move_files_for_document directly, which the original plan did not mention.
• Task 6's test setup was corrected — a correspondent-only assignment doesn't exercise the guard (no m2m_changed fires), so the test as originally written would have passed vacuously, guard or no guard. It now requires a combined tag + correspondent/storage-path assignment, matching the shape of #13178's own regression test.
• Task 7 was rescoped from "write a new regression test for #12386" to "confirm #13178's existing regression tests still pass unchanged" — writing a new test asserting the same final-state outcome would have duplicated coverage that already exists and predates this plan's completion.
• Net effect on scope: this plan still fully replaces use_overrides dual-mode branching and the original_file parameter plumbing (real, unfixed structural debt) with a cleaner WorkflowRunContext abstraction. The ContextVar guard piece is now justified as consolidating two independent point-fixes into one general mechanism, not as fixing an active bug — worth doing, but not urgent.