mirror of
https://github.com/paperless-ngx/paperless-ngx.git
synced 2026-05-06 06:35:24 +00:00
Trenton H
e822e72964
* Replaces loaddata with streaming bulk_create
Replaces call_command('loaddata') with a streaming implementation that
reads manifest records one at a time via ijson, accumulates per-model
batches up to --batch-size, and flushes via bulk_create. This reduces
peak memory and no longer scales directly with the size of the import.
* fix(importer): avoid guardian lru_cache poisoning; include M2M through tables in check_constraints
clear_cache() inside the import transaction emptied Django's ContentType
manager cache while fixture PKs were live, causing downstream ContentType
lookups to repopulate guardian's separate @lru_cache(None) with
fixture-PK objects. After the TestCase transaction rolled back to
original PKs, guardian's lru_cache held stale fixture ContentType
objects, causing MixedContentTypeError in unrelated subsequent tests.
Remove clear_cache() since it was defending against a theoretical
stale-cache scenario that doesn't occur in a proper same-install restore.
Fix check_constraints() to explicitly include auto-created M2M through
tables (populated by .set() after bulk_create) alongside the model tables,
addressing the gap where join-table FK violations would have gone
undetected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* Excludes the consumer and AnonymousUser from any models which might have a FK relation to it. This prevents orphan things like UI setting, which have a relation to no existing user
* Splits into more sub functions for Sonar
* Improvements to the typing of the new functions
* Coverage for some error cases, and removes handling for pk only models. No need to support these
* Final coverage gaps
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>