The consumer's storage phase opens its own file move scope, which nests under the consume task's scope. Moves requested while storing the document therefore stayed pending until the task ended, and the post-consume script could be handed a path from before the move.
The consumer now flushes the pending file moves once after the storage block, outside the media lock and the transaction, and before the post-consume script runs.
check_scheduled_workflows runs the workflow once per matched document inside the task-wide file move scope, so every rename it triggered was deferred to the end of the task. A long run killed by the Celery time limit would then discard all of the renames.
The task now flushes the pending file moves after each document's workflow run and websocket notification, the same way bulk_update_documents does, so renames land document by document while the scope stays open.
document_renamer sent a fake post_save per document to trigger the move, and document_retagger moved a document once for each field it assigned.
The renamer calls move_files_for_document directly for each document, so progress reflects real work. The retagger opens a file move scope per document, so each document is moved once after all of its assignments.
Passing task_cls to Celery replaces the Django fixup's default DjangoTask, so FileMoveScopedTask, built on the plain Task, left every task without delay_on_commit. The workflow AI suggestions trigger failed with an AttributeError.
FileMoveScopedTask now subclasses DjangoTask, which only needs django.db.transaction and is safe to import before apps load. The task scope test also asserts tasks are DjangoTask instances.
Consuming a new version locked the root document row with select_for_update and then waited on media.lock to place the file, the same lock ordering that let concurrent saves deadlock. On MariaDB even the version row's foreign key check holds a shared lock on the root.
The consumer's storage phase now takes media.lock before opening its transaction and runs inside its own file move scope. File placement and workflow email actions inside the transaction re-enter the lock instead of waiting, and filename follow-ups run once after commit.
Emptying the trash deleted documents inside Django's delete transaction, and the post_delete file cleanup waited on media.lock while the deleted rows were locked. A concurrent file move holding media.lock and updating one of those rows could deadlock with it.
empty_trash now holds media.lock around the delete, so the cleanup handler re-enters the lock instead of waiting while rows are locked.
The archiver updated the document row inside a transaction and then waited on media.lock. A concurrent file move holding media.lock and updating the same row could deadlock with it, and the unique archive filename was generated outside the lock from an instance loaded before the parse.
The archiver now acquires media.lock before opening its transaction, refreshes the document under the lock, and moves the archive and thumbnail inside that block, so it never waits on media.lock while holding the row.
Tasks such as bulk_update_documents moved a document's files once for its manual post_save and again for every save its workflows made, each time taking the global media lock.
All tasks now run through FileMoveScopedTask, which opens a file move scope around the task body, including when the task raises. bulk_update_documents flushes after each document so long bulk edits still make steady progress.
A tag PATCH moved files from inside the tags.set() transaction while holding the document row lock. A concurrent save holding media.lock and waiting on that row deadlocked with it, and every later save on the instance queued behind media.lock until the webserver restarted.
FileMoveMiddleware opens a file move scope around every request, in sync or async mode, so each changed document is moved once after the view returns and never while a transaction holds row locks.
Each save of a document moved its files on the spot, so one logical change could move them several times, and moves could run inside another open transaction.
defer_file_moves() opens a ContextVar-backed scope. Inside it, move requests are recorded and the outermost scope moves each document once on exit, or on commit if a transaction is still open. A failing move is logged and does not stop the others. Outside any scope moves stay immediate.
The file move receiver refreshed the caller's own instance from the database, which wiped unsaved in-memory changes. It ran on every m2m_changed action including pre_add and pre_remove, wrote modified even when nothing moved, and its no-move branch skipped trashed documents.
The move body now lives in documents.file_moves.move_files_for_document. It skips documents without a filename before taking media.lock, then takes the lock with a 300 second timeout, loads a fresh instance by pk, writes only when a filename column changes, and returns the instance. The receiver ignores reverse and pre_* m2m actions and copies just the final filename columns back onto the caller's instance.
Every caller built its own FileLock on media.lock, so code already holding it could not call into code that takes it again without blocking itself. That made it impossible to take media.lock before opening a transaction in paths that also attach files or move files inside that transaction.
media_lock() hands out one FileLock instance per path with per-thread hold counts, so a thread re-enters while other threads and processes stay excluded. All media.lock call sites now use it.
A full save() of a Document instance loaded before another process moved its file wrote the old filename back. The DB then pointed at a path that no longer existed, and recovery only worked when the recomputed path happened to match the file's real location.
Document.save() now leaves filename and archive_filename out of full saves on existing rows. Writers that own those columns opt in with write_file_paths=True or an explicit update_fields. A full save of a row deleted elsewhere now raises instead of re-inserting it.
turbohtml 1.10.0 -> 1.15.3. Includes sanitizer hardening (GHSA-9x48-74fj-x342, GHSA-wh2c-9vrv-w597, GHSA-pjrh-5m66-3wwf, URL schemes hidden behind character references in the bleach shim). Only the bleach clean shim and linkify are used, in the mail parser.
* Fix: match Apple Mail tag colors case-insensitively
* Fix: reject OAuth callbacks with an unrecognized scope
* Fix: refuse to refresh OAuth tokens for non-OAuth accounts or missing refresh tokens
* Fix: parse datetime strings in the format_datetime filter and reject invalid ones
* Fix: reject a PAPERLESS_URL without a hostname
* Fix: report a database error in the system status instead of failing with a 500
* Fix: surface the database error when refreshing a document before moving its files
* Fix: report no unapplied migrations when the applied migrations cannot be read
* Fix: return the parsed value directly in format_datetime for typing
* Chore: annotate the paperless_dirs fixture in the refresh failure test
* Allow null for the latest_migration system status type
---------
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
stumpylog
github-actions[bot]andshamoon
dependabot[bot]