Commit Graph

2119 Commits

Author SHA1 Message Date
Michael Eischer 278e457e1f data: use data.TreeWriter to serialize&write data.Tree
Always serialize trees via TreeJSONBuilder. Add a wrapper called
TreeWriter which combines serialization and saving the tree blob in the
repository. In the future, TreeJSONBuilder will have to upload tree
chunks while the tree is still serialized. This will a wrapper like
TreeWriter, so add it right now already.

The archiver.treeSaver still directly uses the TreeJSONBuilder as it
requires special handling.
2026-01-31 19:18:36 +01:00
Winfried Plappert 86ccc6d445 Bugfix: restic check: add missing finalizeSnapshotFilter() (#5644)
add missing finalizeSnapshotFilter() to cmd.RunE()

---------

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2026-01-26 21:08:15 +00:00
Michael Eischer a8be8e36fa Merge pull request #5621 from MichaelEischer/copy-stream-snapshots
copy: iterate through snapshots
2025-12-03 21:21:05 +01:00
Michael Eischer 1e6ed458ff remove old // +build comments 2025-11-30 11:53:23 +01:00
Winfried Plappert ce57961f14 restic check with snapshot filters (#5469)
---------

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2025-11-28 19:12:38 +00:00
Michael Eischer e1bc2fb71a copy: iterate through snapshots 2025-11-26 22:48:54 +01:00
Michael Eischer 046b0e711d repository: add SaveBlobAsync method 2025-11-26 21:18:21 +01:00
Michael Eischer 134893bd35 copy: use AssociatedBlobSet to keep track of processed trees 2025-11-26 21:00:18 +01:00
Michael Eischer 46ebee948f stats: use AssociatedBlobSet 2025-11-26 20:59:39 +01:00
Michael Eischer d91fe1d7e1 diff: use AssociatedBlobSet 2025-11-26 20:59:39 +01:00
Michael Eischer ff099a216a copy: use AssociatedBlobSet 2025-11-26 20:59:38 +01:00
Michael Eischer 7e80536a9b Merge pull request #5472 from wplapper/cmd_copy_stream
restic copy --stream: run one large copy operation crossing snapshot boundaries - issue #5453
2025-11-26 20:57:46 +01:00
Michael Eischer f9e5660e75 output which source and target snapshot belong together 2025-11-23 22:01:53 +01:00
Michael Eischer e79b01d82f more aggressive batching 2025-11-23 21:46:03 +01:00
Michael Eischer 857b42fca4 merge into existing copy test 2025-11-23 19:08:49 +01:00
Michael Eischer 39db78446f Simplify test 2025-11-23 19:05:55 +01:00
Michael Eischer cf409b7c66 automatically batch snapshots in copy 2025-11-23 17:40:37 +01:00
Michael Eischer f95dc73d38 deduplicate blob enqueuing 2025-11-23 17:13:10 +01:00
Michael Eischer 63bc1405ea unify snapshot copy codepaths 2025-11-23 17:12:54 +01:00
Michael Eischer 05364500b6 use correct context 2025-11-23 16:25:09 +01:00
Michael Eischer e775192fe7 don't sort snapshots, drop duplicate code and cleanup copyTreeBatched function signature 2025-11-23 16:20:40 +01:00
Michael Eischer 4395a77154 copy: remove bugous seenBlobs set 2025-11-23 16:06:45 +01:00
Michael Eischer 81d8bc4ade repository: replace CopyBlobs with Repack implementation 2025-11-23 16:06:29 +01:00
Michael Eischer 6174c91042 Merge pull request #5588 from seqizz/g_timezoneshow
snapshots: Show timezone in non-compact output
2025-11-19 22:06:37 +01:00
Winfried Plappert b87f7586e4 restic copy --batch: a fresh start from commit 382616747
Instead of rebasing my code, I decided to start fresh, since WithBlobUploader()
has been introduced.

changelog/unreleased/issue-5453:
doc/045_working_with_repos.rst:
the usual

cmd/restic/cmd_copy.go:
gather all snaps to be collected - collectAllSnapshots()
run overall copy step - func copyTreeBatched()
helper copySaveSnapshot() to save the corresponding snapshot

internal/repository/repack.go:
introduce wrapper CopyBlobs(), which passes parameter `uploader restic.BlobSaver` from
WithBlobUploader() via copyTreeBatched() to repack().

internal/backend/local/local_windows.go:
I did not touch it, but gofmt did: whitespace
2025-11-19 07:09:24 +00:00
Gürkan dc4e9b31f6 snapshots: Show timezone in non-compact output 2025-11-18 13:32:44 +01:00
Michael Eischer 8767549367 Merge pull request #5601 from MichaelEischer/snapshots-fix-groupby-with-latest
snapshots: correctly handle --latest in combination with --group-by
2025-11-17 22:50:50 +01:00
Michael Eischer 5afe61585b snapshots: correctly handle --latest in combination with --group-by 2025-11-17 22:26:57 +01:00
Michael Eischer 0ff3e20c4b prune: return proper error if blob cannot be found 2025-11-16 17:04:03 +01:00
Michael Eischer 3b854d9c04 Merge pull request #5449 from provokateurin/restore-ownership-by-name
feat(internal/fs/node): Restore ownership by name
2025-11-16 16:50:36 +01:00
provokateurin 8fae46011a feat(internal/fs/node): Restore ownership by name 2025-11-16 16:40:58 +01:00
Winfried Plappert 25611f4628 restic copy - add statistics counters
cmd/restic/cmd_copy.go:
add function copyStats() and call it before the actual copying starts.

changelog/unreleased/pull-5319:
rephrased wording of the statistics counters.
2025-11-16 13:47:10 +01:00
Winfried Plappert 90ac3efa88 restic copy - add additional status counters
'copyTree()' now counts and sizes the blobs in 'copyBlobs' and prints them out
via 'Verbosef()'.
2025-11-16 13:46:27 +01:00
Michael Eischer 14f3bc8232 Merge pull request #5560 from MichaelEischer/index-iterators
index: port to  modern Go iterators
2025-11-16 13:24:48 +01:00
Michael Eischer 38c543457e index: convert to implement modern go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer a0925fa922 repository: set progress bar maximum in Repack 2025-11-16 12:51:46 +01:00
Michael Eischer b2afccbd96 repository: remove unused obsoletePacks return values from Repack 2025-11-16 12:51:46 +01:00
Michael Eischer 0624b656b8 Merge pull request #5558 from MichaelEischer/simplify-blob-upload
repository: enforce correct usage of SaveBlob
2025-11-16 12:51:01 +01:00
Michael Eischer 157f174dd9 Merge pull request #5370 from hashier/feat/exclude-macOS-cloud-files
feat(backup): add possibility to exclude macOS cloud-only files
2025-11-16 11:57:37 +01:00
Michael Eischer c6e33c3954 repository: enforce that SaveBlob is called within WithBlobUploader
This is achieved by removing SaveBlob from the public API and only
returning it via a uploader object that is passed in by
WithBlobUploader.
2025-10-12 18:26:26 +02:00
Michael Eischer aa7bd241d9 init: move more logic into global package 2025-10-12 18:08:26 +02:00
Michael Eischer a816b827cf extract GlobalOptions into internal/global package
Rough steps:
```
mv cmd/restic/global* cmd/restic/secondary_repo* internal/global/
sed -i "s/package main/package global/" internal/global/*.go
Rename "GlobalOptions" to "Options" in internal/global/
Replace everywhere " GlobalOptions" -> " global.Options"
Replace everywhere "\*GlobalOptions" -> " *global.Options"
Make SecondaryRepoOptions public
Make create public
Make version public
```
2025-10-12 17:56:28 +02:00
Michael Eischer 2c677d8db4 global: make private fields public 2025-10-12 17:56:28 +02:00
Michael Eischer 394c8de502 add package to create a prepopulated backend registry 2025-10-12 17:56:28 +02:00
Michael Eischer a632f490fa Merge pull request #5550 from MichaelEischer/refactor-check-data-selection
check: refactor pack selection for read data
2025-10-12 17:51:00 +02:00
Michael Eischer ac4642b479 repository: replace StartPackUploader+Flush with WithBlobUploader
The new method combines both step into a single wrapper function. Thus
it ensures that both are always called in pairs. As an additional
benefit this slightly reduces the boilerplate to upload blobs.
2025-10-08 22:49:45 +02:00
Srigovind Nayak 481fcb9ca7 backup: return exit code 3 if not all targets are available (#5347)
to make the exit code behaviour consistent with files inaccessible during the backup phase, making this change to exit with code 3 if not all target files/folders are accessible for backup

---------

Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2025-10-05 15:38:52 +02:00
Srigovind Nayak 22f254c9ca feat: allow override env RESTIC_HOST with flag to filter all snapshots (#5541) 2025-10-05 13:22:50 +02:00
Christopher Loessl f3d95893b2 feat(backup): add possibility to exclude macOS cloud-only files 2025-10-04 19:22:51 +02:00
Winfried Plappert a2a49cf784 list integration test: error scanning 'restic list blobs' (#5311)
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
2025-10-04 12:18:32 +00:00