Commit Graph
11679 Commits
Author SHA1 Message Date
stumpylogandClaude Sonnet 5 1ea9775ab4 Refactor: route document_exporter through ExportSink, direct-to-zip
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-23 11:53:45 -07:00
stumpylog 02b97fbaaf Fix: de-duplicate source_file fixture across sink test classes
TestDirectoryExportSink and TestZipExportSink each defined an
identical source_file fixture; hoist it to module scope.
2026-07-22 15:38:55 -07:00
stumpylog 0e538acdef Feature: add ZipExportSink with atomic finalize and manifest spooling 2026-07-22 15:36:50 -07:00
stumpylog 7f58d16758 Fix: make ExportSink a real ABC per design spec
The implementation plan for Task 2 diverged from the design spec
(export-sink-architecture-design.md), leaving ExportSink as a plain
class with NotImplementedError bodies instead of the specified
AbstractContextManager subclass. Use abc.ABC + @abstractmethod so a
concrete sink missing a required method fails at instantiation
rather than at first call.
2026-07-22 15:07:24 -07:00
stumpylog 344414bb47 Feature: add ExportSink ABC and DirectoryExportSink 2026-07-22 14:57:48 -07:00
stumpylog 8263f75643 Feature: add export package with StreamingManifestWriter and _dumps 2026-07-22 14:34:55 -07:00
stumpylog 72d031a7e2 Docs: remove specs/plans unrelated to direct-to-zip export
This branch is scoped to the export sink architecture + zip
compression control work; drop the search, bulk-edit,
ingestion-staging, vector-store, and other unrelated design docs
and scratch profiling scripts that had accumulated on it.
2026-07-22 14:26:08 -07:00
stumpylogandClaude Opus 4.8 1a253d0f26 Docs: ingestion staging & enqueue unification implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 dc4cb829a6 Docs: revise ingestion staging spec per critical review
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 cad2d2d151 Docs: ingestion staging & enqueue unification design spec
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 8db09d7c92 Docs: bulk-edit operation registry spec + implementation plan
Adds the extensibility design that hoists the bulk-edit operation
definition (today smeared across serialisers.py, views.py, bulk_edit.py
and keyed three different ways) behind a BulkEditOperation registry +
PermissionRequirements value object, with per-operation OpenAPI examples.
Contract-preserving refactor; both docs reviewed across multiple passes
(permission matrix verified, both view call sites accounted for).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 b02a151e45 Docs: export zip compression implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 0083443e77 Docs: verify zstd/3.14 facts in compression spec (empirical, uv --python 3.14)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 2cd9b94c43 Docs: export sink architecture implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 b76bd9a7bb Docs: revise export specs per critical review
- Preserve --zip --delete (real tested feature), only --compare-* hard-error
- Mandate .as_posix() arcnames; --data-only, SCRATCH_DIR, atomicity-honesty notes
- Progress tracking stays in command; sinks decoupled from PaperlessCommand
- Compression spec: gate/verify zstd, fix import failure mode, patchable probe seam

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 87a9fcadc8 Docs: export sink architecture + zip compression design specs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 1c4f6b6144 Docs(beta): flesh out search error-shapes spec and implementation plan
Replace the search-error-shapes stub with a full design spec and a TDD
implementation plan for friendlier advanced-search error messages.
Empirically validated against a live Tantivy index: three error families
(UnknownFieldError, InvalidFieldValueError, MalformedQueryError),
proactive numeric validation plus a parse_query backstop, comparison
operators confirmed working, and a parse-based field drift guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylog 2d985961b0 Fture search error ideas 2026-07-22 14:17:28 -07:00
Trenton Holmes 08a2186de1 Tracks the tanvity update blocker 2026-07-22 14:17:28 -07:00
Trenton Holmes a89fdfb033 Tracks the tanvity update blocker 2026-07-22 14:17:28 -07:00
Trenton Holmes c0fc6c0475 More done work 2026-07-22 14:17:27 -07:00
Trenton Holmes 28f291c503 Store more profiling files 2026-07-22 14:17:27 -07:00
stumpylog 8263460b7b Removes these for starting implementation 2026-07-22 14:17:27 -07:00
stumpylog f4e997e524 Marks some things as done 2026-07-22 14:17:27 -07:00
stumpylog 1944208de4 Storing more stuff for stuff 2026-07-22 14:17:27 -07:00
Trenton Holmes db14bd6b61 Storing more ideas/plans 2026-07-22 14:17:27 -07:00
stumpylog f41181f992 Additional ideas and plans 2026-07-22 14:17:27 -07:00
stumpylog b4729ec5d2 Minor plan updates 2026-07-22 14:17:26 -07:00
stumpylog 3cd772d8e9 Just for later ideas, store some brainstorming sessions with Claude 2026-07-22 14:17:26 -07:00
github-actions[bot]GitHubgithub-actions <41898282+github-actions[bot]@users.noreply.github.com>shamoonstumpylog
1203203e08 Documentation: Add v3.0.0 changelog (#13199)
---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-22 13:05:33 -07:00
shamoon 6cf3c47814 Documentation: fix a couple broken links 2026-07-22 11:37:56 -07:00
shamoon 24aee98ad5 Documentation: fix v3 migration guide header level 2026-07-22 11:31:15 -07:00
shamoonandGitHub 2ab1ac7cb8 [Beta] Paperless-ngx v3.0.0 Beta (#12713) v3.0.0 2026-07-22 10:55:13 -07:00
shamoon c2d9c52cde Merge branch 'main' into beta 2026-07-22 10:30:57 -07:00
cdb11b94b5 New Crowdin translations by GitHub Action (#13153)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-22 10:30:33 -07:00
GitHub Actions cfd7a3a7f0 Auto translate strings 2026-07-22 17:18:01 +00:00
shamoon c6329f4b76 Merge branch 'beta' into dev 2026-07-22 10:16:33 -07:00
shamoonandGitHub 5e06987102 Chore (beta): mark first batch of thumbnails as priorty for LCP warning (#13196) 2026-07-22 10:15:56 -07:00
shamoonandGitHub 3f93557934 Fix (beta): prevent pdfjs offsetParent warning (#13197) 2026-07-22 10:06:41 -07:00
shamoon afe6ad2192 Chore: remove unused import 2026-07-22 09:23:18 -07:00
GitHub Actions a3f9fa0369 Auto translate strings 2026-07-22 15:45:25 +00:00
shamoonandGitHub e106618c39 Fix (beta): dont include hidden buttons in virtual scroll container height (#13194) 2026-07-22 08:42:31 -07:00
shamoonandGitHub d113826cf8 Tweakhancement: allow some bulk operations with all (#13193) 2026-07-22 08:28:49 -07:00
shamoonandGitHub c063b3799f Fix (beta): fix custom field bulk editing for 'all' (#13191) 2026-07-22 08:14:53 -07:00
81795bc93a Security (beta): enforce current permissions in autocomplete (#13188)
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-22 08:07:43 -07:00
shamoonandGitHub 6664f05543 Security (beta): bound email linkification (#13187) 2026-07-22 07:41:12 -07:00
shamoonandGitHub 8aab34ea73 Security (beta): two small permission check fixes (#13186) 2026-07-21 16:50:28 +00:00
8404198ec8 Fix (beta): short-circuit ObjectOwnedOrGrantedPermissionsFilter for superusers (#13183)
Mirrors the existing short-circuit already present in its sibling class,
ObjectOwnedPermissionsFilter, which this one lacked.

Benchmarked as a superuser against a 100k-document corpus: no measurable
difference. django-guardian's own get_objects_for_user() (called via
super().filter_queryset()) already returns the queryset unrestricted for
superusers before this change, so the guardian permission tables were
never actually queried for superusers either way. This change only skips
building a redundant 3-way OR of conditions that are all subsets of the
already-unrestricted queryset -- worth keeping for clarity/consistency
with the sibling filter, not as a performance claim.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-21 07:31:58 -07:00
d8d8872414 Fix (beta): compute num_notes via a subquery instead of Count()+distinct() (#13182)
DocumentViewSet.get_queryset() annotated num_notes via a LEFT JOIN to
documents_note plus Count(), which requires the database to aggregate
every matching document's note count before it can even sort or paginate
the result -- on every list request, not just filtered ones.

Switched to the same correlated-subquery pattern already used two lines
above for effective_content (Subquery + OuterRef), which Django compiles
to portable SQL across sqlite/postgresql/mariadb rather than a join-based
aggregate.

Benchmarked against a 100k synthetic document corpus (Postgres): the
plain document list request dropped from ~2.2-2.4s to ~0.9-1.0s. Note
that a separate, larger cost remains in the same queryset's plain
.distinct() call, which still forces a full sort over every matching
row regardless of this fix -- tracked separately, not addressed here.

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-20 13:59:44 -07:00
shamoonandGitHub ff609c2987 Change: update root modified timestamp on version changes (#13170) 2026-07-20 15:45:24 +00:00