Commit Graph
11689 Commits
Author SHA1 Message Date
stumpylog 50dec14653 Test: assert compression forwarding via call args, not a real archive
test_compression_method_is_applied_to_file_entries built a real zip and
read back compress_type — but that only confirms zipfile.ZipFile()
honors its own compression= kwarg, which is documented stdlib behavior,
not something our code could get wrong. ZipExportSink's only
responsibility is forwarding compression/compresslevel unchanged to
ZipFile(); assert that directly against the mocked constructor call.
2026-07-23 14:26:01 -07:00
stumpylog cc4917efec Test: remove test asserting Python's own compression behavior
test_compressing_method_beats_stored asserted that DEFLATED produces a
smaller archive than STORED — that's zlib's job, not ours.
test_compression_method_is_applied_to_file_entries already covers what
our code is actually responsible for: threading the requested
compression method through to the zip entry's compress_type.
2026-07-23 14:06:37 -07:00
stumpylog 672fc6a0d4 Feature: add --zip-compression and --zip-compression-level flags 2026-07-23 14:02:41 -07:00
stumpylog 943d322196 Feature: ZipExportSink accepts compression method and level 2026-07-23 13:52:55 -07:00
stumpylog 4d956fa13b Feature: add export compression policy module 2026-07-23 13:27:06 -07:00
stumpylog 3963581af0 Chore: remove completed export-sink-architecture plan/spec
The design is now implemented (src/documents/export/sinks.py); the
planning docs served their purpose. The deferred zip-compression-control
follow-up plan/spec remains, since that work hasn't landed yet.
2026-07-23 13:01:42 -07:00
stumpylog 581fbab8f3 Polish: atomic zip commit via Path.replace, widen sink params to ExportSink
Path.rename() raises FileExistsError on Windows when the destination
already exists; Path.replace() is atomic cross-platform. Also widen
document_exporter's sink parameters from the concrete
DirectoryExportSink | ZipExportSink union to the ExportSink ABC, so a
future sink implementation is a pure addition rather than requiring
every call site's annotation to change.
2026-07-23 12:59:27 -07:00
stumpylog 7044d8ae5f Refactor: de-duplicate BLAKE2b compare and simplify zip dir-marker loop
DirectoryExportSink.add_json and _commit_streamed_file each inlined the
same hash-and-compare logic; extracted _content_unchanged(). Replaced
ZipExportSink._ensure_dirs's repeated slice/join with a prefix
accumulator and hoisted the _zip-is-open assertion out of the loop. No
behavioral change.
2026-07-23 12:27:15 -07:00
stumpylog 5e21ba891d Fix: annotate ExportSink.stream's return type for pyrefly
The abstract method had no return annotation, so pyrefly inferred -> None
and flagged both DirectoryExportSink.stream and ZipExportSink.stream as
incompatible overrides.
2026-07-23 12:20:29 -07:00
stumpylog c2c30541bb Test: guard --zip combined with --compare-* flags 2026-07-23 12:04:01 -07:00
stumpylogandClaude Sonnet 5 1ea9775ab4 Refactor: route document_exporter through ExportSink, direct-to-zip
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-23 11:53:45 -07:00
stumpylog 02b97fbaaf Fix: de-duplicate source_file fixture across sink test classes
TestDirectoryExportSink and TestZipExportSink each defined an
identical source_file fixture; hoist it to module scope.
2026-07-22 15:38:55 -07:00
stumpylog 0e538acdef Feature: add ZipExportSink with atomic finalize and manifest spooling 2026-07-22 15:36:50 -07:00
stumpylog 7f58d16758 Fix: make ExportSink a real ABC per design spec
The implementation plan for Task 2 diverged from the design spec
(export-sink-architecture-design.md), leaving ExportSink as a plain
class with NotImplementedError bodies instead of the specified
AbstractContextManager subclass. Use abc.ABC + @abstractmethod so a
concrete sink missing a required method fails at instantiation
rather than at first call.
2026-07-22 15:07:24 -07:00
stumpylog 344414bb47 Feature: add ExportSink ABC and DirectoryExportSink 2026-07-22 14:57:48 -07:00
stumpylog 8263f75643 Feature: add export package with StreamingManifestWriter and _dumps 2026-07-22 14:34:55 -07:00
stumpylog 72d031a7e2 Docs: remove specs/plans unrelated to direct-to-zip export
This branch is scoped to the export sink architecture + zip
compression control work; drop the search, bulk-edit,
ingestion-staging, vector-store, and other unrelated design docs
and scratch profiling scripts that had accumulated on it.
2026-07-22 14:26:08 -07:00
stumpylogandClaude Opus 4.8 1a253d0f26 Docs: ingestion staging & enqueue unification implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 dc4cb829a6 Docs: revise ingestion staging spec per critical review
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 cad2d2d151 Docs: ingestion staging & enqueue unification design spec
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 8db09d7c92 Docs: bulk-edit operation registry spec + implementation plan
Adds the extensibility design that hoists the bulk-edit operation
definition (today smeared across serialisers.py, views.py, bulk_edit.py
and keyed three different ways) behind a BulkEditOperation registry +
PermissionRequirements value object, with per-operation OpenAPI examples.
Contract-preserving refactor; both docs reviewed across multiple passes
(permission matrix verified, both view call sites accounted for).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 b02a151e45 Docs: export zip compression implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 0083443e77 Docs: verify zstd/3.14 facts in compression spec (empirical, uv --python 3.14)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 2cd9b94c43 Docs: export sink architecture implementation plan
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 b76bd9a7bb Docs: revise export specs per critical review
- Preserve --zip --delete (real tested feature), only --compare-* hard-error
- Mandate .as_posix() arcnames; --data-only, SCRATCH_DIR, atomicity-honesty notes
- Progress tracking stays in command; sinks decoupled from PaperlessCommand
- Compression spec: gate/verify zstd, fix import failure mode, patchable probe seam

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 87a9fcadc8 Docs: export sink architecture + zip compression design specs
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylogandClaude Opus 4.8 1c4f6b6144 Docs(beta): flesh out search error-shapes spec and implementation plan
Replace the search-error-shapes stub with a full design spec and a TDD
implementation plan for friendlier advanced-search error messages.
Empirically validated against a live Tantivy index: three error families
(UnknownFieldError, InvalidFieldValueError, MalformedQueryError),
proactive numeric validation plus a parse_query backstop, comparison
operators confirmed working, and a parse-based field drift guard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 14:17:28 -07:00
stumpylog 2d985961b0 Fture search error ideas 2026-07-22 14:17:28 -07:00
Trenton Holmes 08a2186de1 Tracks the tanvity update blocker 2026-07-22 14:17:28 -07:00
Trenton Holmes a89fdfb033 Tracks the tanvity update blocker 2026-07-22 14:17:28 -07:00
Trenton Holmes c0fc6c0475 More done work 2026-07-22 14:17:27 -07:00
Trenton Holmes 28f291c503 Store more profiling files 2026-07-22 14:17:27 -07:00
stumpylog 8263460b7b Removes these for starting implementation 2026-07-22 14:17:27 -07:00
stumpylog f4e997e524 Marks some things as done 2026-07-22 14:17:27 -07:00
stumpylog 1944208de4 Storing more stuff for stuff 2026-07-22 14:17:27 -07:00
Trenton Holmes db14bd6b61 Storing more ideas/plans 2026-07-22 14:17:27 -07:00
stumpylog f41181f992 Additional ideas and plans 2026-07-22 14:17:27 -07:00
stumpylog b4729ec5d2 Minor plan updates 2026-07-22 14:17:26 -07:00
stumpylog 3cd772d8e9 Just for later ideas, store some brainstorming sessions with Claude 2026-07-22 14:17:26 -07:00
github-actions[bot]GitHubgithub-actions <41898282+github-actions[bot]@users.noreply.github.com>shamoonstumpylog
1203203e08 Documentation: Add v3.0.0 changelog (#13199)
---------

Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: shamoon <4887959+shamoon@users.noreply.github.com>
Co-authored-by: stumpylog <797416+stumpylog@users.noreply.github.com>
2026-07-22 13:05:33 -07:00
shamoon 6cf3c47814 Documentation: fix a couple broken links 2026-07-22 11:37:56 -07:00
shamoon 24aee98ad5 Documentation: fix v3 migration guide header level 2026-07-22 11:31:15 -07:00
shamoonandGitHub 2ab1ac7cb8 [Beta] Paperless-ngx v3.0.0 Beta (#12713) v3.0.0 2026-07-22 10:55:13 -07:00
shamoon c2d9c52cde Merge branch 'main' into beta 2026-07-22 10:30:57 -07:00
cdb11b94b5 New Crowdin translations by GitHub Action (#13153)
Co-authored-by: Crowdin Bot <support+bot@crowdin.com>
2026-07-22 10:30:33 -07:00
GitHub Actions cfd7a3a7f0 Auto translate strings 2026-07-22 17:18:01 +00:00
shamoon c6329f4b76 Merge branch 'beta' into dev 2026-07-22 10:16:33 -07:00
shamoonandGitHub 5e06987102 Chore (beta): mark first batch of thumbnails as priorty for LCP warning (#13196) 2026-07-22 10:15:56 -07:00
shamoonandGitHub 3f93557934 Fix (beta): prevent pdfjs offsetParent warning (#13197) 2026-07-22 10:06:41 -07:00
shamoon afe6ad2192 Chore: remove unused import 2026-07-22 09:23:18 -07:00