Commit Graph

244 Commits

Author SHA1 Message Date
Michael Eischer 7d36449ea8 repository: change LoadBlob, LookupBlob and LookupBlobSize to BlobHandles (#21857) 2026-06-14 15:30:00 +02:00
Michael Eischer 8e11f5747d restic: introduce Counter interface to decouple from ui/progress (#21861)
decouple restic and ui/progress packages
2026-06-13 21:08:18 +02:00
Michael Eischer c8a0bc2c5e repository: move crypto package to internal/repository/crypto 2026-06-13 20:05:06 +02:00
Michael Eischer 1acc19dac5 repository: unexport SearchKey 2026-06-13 19:49:03 +02:00
Michael Eischer d62e42e620 repository: unexport NewChecker 2026-06-13 19:48:02 +02:00
Michael Eischer c062a78dcd repository: move Blob, Blobs and PackedBlob to pack package
This removes them from the public interface. The latter now only
provides the PackBlob interface, without being bound to the type used
internally by the pack package.
2026-06-13 18:58:37 +02:00
Michael Eischer 35af104749 restic: change LookupBlob to return []PackBlob 2026-06-13 17:54:58 +02:00
Michael Eischer ccb5ae1592 restic: change ListBlobs to return PackBlob
PackBlob is a limited interface that only exposes a part of the
information provided by PackedBlob. Most of the changes are switches
from direct value lookups to the interface methods, with a few larger
changes to let the tests still work.
2026-06-13 17:54:38 +02:00
Michael Eischer c060c317d3 repository: unexport listPacksFromIndex
`ListPacksFromIndex` only has repository-internal callers left (besides
test code).
2026-06-13 17:54:03 +02:00
Michael Eischer a9e0b46358 restic: list pack header via ListPackHandles
Replace ListPack with ListPackHandles so callers only receive blob
handles from pack headers, not layout fields.
2026-06-13 17:50:21 +02:00
Michael Eischer 8169814b38 restic: switch LoadBlobsFromPack to BlobHandles
LoadBlobsFromPack now resolves the handles to Blobs. Repository internal
code can still use the Blob-based method.

The loader used in the filerestorer test now has to implement sorting
the blobs by offset itself as it no longer has access to the repository-internal
dataypes.
2026-06-13 17:50:21 +02:00
Michael Eischer abfc9bbdff Merge pull request #21868 from MichaelEischer/fix-zero-sized-blobs
repository: fix zero-sized blobs in v2 repos
2026-06-11 20:43:38 +02:00
Michael Eischer c745f810b3 backend/cache: use backend types not restic types (#21860) 2026-06-10 22:45:39 +02:00
Michael Eischer 0fc7444e32 repository: fix zero-sized blobs in v2 repos 2026-06-10 22:03:40 +02:00
Michael Eischer 7d25ca9d67 repository: omit unused headerSize from ListPack() 2026-05-31 22:53:00 +02:00
Michael Eischer f86307d223 Merge pull request #21827 from MichaelEischer/fix-pack-repair
repair packs: correctly handle packs with missing/incomplete index entry
2026-05-31 15:57:50 +02:00
Michael Eischer c221cd06ad repository: forget id of unreadable packs on index rebuild (#21826) 2026-05-30 22:09:33 +02:00
Michael Eischer c669cc7a7d introduce restic.Blobs type with sort method 2026-05-30 10:10:39 +02:00
Michael Eischer 4c0dc9e202 index: support incremental index loading
Do not require a full index reload if only a few additional index files
have been added. This can drastically speed up loading the index in the
mount command.
2026-05-07 22:52:03 +02:00
Michael Eischer d1937a530b clarify pack ID in decryption error (#5710)
pack ID is included in full. In addition, the error message now says
that it is a pack file.
2026-02-18 20:43:10 +01:00
Michael Eischer f84d398989 repository: prevent test deadlock within WithBlobUploader
Calling t.Fatal internally triggers runtime.Goexit . This kills the
current goroutine while only running deferred code. Add an extra context
that gets canceled if the go routine exits while within the user
provided callback.
2026-01-31 19:18:36 +01:00
Michael Eischer 5607fd759f repository: fix race condition for blobSaver shutdown
wg.Go() may not be called after wg.Wait(). This prevents connecting two
errgroups such that the errors are propagated between them if the child
errgroup dynamically starts goroutines. Instead use just a single errgroup,
and sequence the shutdown using a sync.WaitGroup. This is far simpler
and does not require any "clever" tricks.
2025-11-26 21:18:22 +01:00
Michael Eischer 046b0e711d repository: add SaveBlobAsync method 2025-11-26 21:18:21 +01:00
Michael Eischer 07d090f233 repository: expose AssociatedBlobSet via repository interface 2025-11-26 20:59:08 +01:00
Michael Eischer cf409b7c66 automatically batch snapshots in copy 2025-11-23 17:40:37 +01:00
Michael Eischer 405813f250 repository: fix LookupBlobSize to also report pending blobs 2025-11-23 17:09:07 +01:00
Michael Eischer 38c543457e index: convert to implement modern go iterators 2025-11-16 12:56:37 +01:00
Michael Eischer c6e33c3954 repository: enforce that SaveBlob is called within WithBlobUploader
This is achieved by removing SaveBlob from the public API and only
returning it via a uploader object that is passed in by
WithBlobUploader.
2025-10-12 18:26:26 +02:00
Michael Eischer ac4642b479 repository: replace StartPackUploader+Flush with WithBlobUploader
The new method combines both step into a single wrapper function. Thus
it ensures that both are always called in pairs. As an additional
benefit this slightly reduces the boilerplate to upload blobs.
2025-10-08 22:49:45 +02:00
Michael Eischer 4426dfe6a9 repository: replace SetIndex method with internal loadIndexWithCallback method 2025-10-03 19:36:57 +02:00
Michael Eischer f0955fa931 repository: add Checker() method to repository to replace unchecked cast 2025-10-03 19:34:33 +02:00
Michael Eischer 52eb66929f repository: deduplicate index progress bar initializaton 2025-10-03 18:55:46 +02:00
Michael Eischer b6c50662da repository: don't ignore cache clearing error 2025-10-03 18:22:42 +02:00
Michael Eischer 4dc71f24c5 backends: pass error logger to backends 2025-10-03 18:22:42 +02:00
Michael Eischer 60d80a6127 Fix linter warnings 2025-09-21 22:24:15 +02:00
Srigovind Nayak f13e9c10a4 Add support for additional compression levels fastest and better (#5321)
* repository: expose addtional compression levels

* adding better and fastest compression levels for zstd

* repository: add changelog entry for issue-4728

* chore: fix golint issues

* chore: sort compression modes in the help text

* updating review comments
2025-03-31 21:21:12 +02:00
Michael Eischer 62453f9356 repository: randomly distribute blobs over two pack files 2025-03-23 12:29:16 +01:00
Martin Smith 092899df8b Remove unnecessary type conversions. 2025-03-22 18:20:30 +00:00
Michael Eischer 69249372bf Merge pull request #5249 from MichaelEischer/fix-repair-index
Prevent creation of oversized indexes and automatically rewrite them.
2025-03-22 14:29:39 +01:00
Michael Eischer c970e58739 backend: refactor backend Connections and HasAtomicReplace into Properties 2025-02-16 22:27:58 +01:00
Michael Eischer 2fd8a3865c index: automatically write full indexes in StorePack 2025-02-16 16:39:38 +01:00
Michael Eischer b7ff8ea9cd repository: expose cache via method 2025-01-13 22:40:18 +01:00
Michael Eischer 99e105eeb6 repository: restrict SaveUnpacked and RemoveUnpacked
Those methods now only allow modifying snapshots. Internal data types
used by the repository are now read-only. The repository-internal code
can bypass the restrictions by wrapping the repository in an
`internalRepository` type.

The restriction itself is implemented by using a new datatype
WriteableFileType in the SaveUnpacked and RemoveUnpacked methods. This
statically ensures that code cannot bypass the access restrictions.

The test changes are somewhat noisy as some of them modify repository
internals and therefore require some way to bypass the access
restrictions. This works by capturing an `internalRepository` or
`Backend` when creating the Repository using a test helper function.
2025-01-13 22:39:57 +01:00
Srigovind Nayak 88174cd0a4 cache: remove redundant index file cleanup
addressing code review comments
2024-08-17 00:21:49 +05:30
Srigovind Nayak b7d014b685 Revert "repository: removed redundant prepareCache method from Repository"
This reverts commit 720609f8ba.
2024-08-17 00:18:13 +05:30
Srigovind Nayak 720609f8ba repository: removed redundant prepareCache method from Repository
* remove the prepareCache method from the Repository
* changed the signature of the SetIndex function to no longer return an error
2024-08-11 23:41:07 +05:30
Michael Eischer ae1cb889dd Add more checks for canceled contexts 2024-07-31 19:30:47 +02:00
Michael Eischer b80aa7b1cc repository: prevent initialization if a snapshot exists 2024-06-14 20:37:01 +02:00
Michael Eischer bab760369f repository: double check that there is not repository before init
Apparently, calling `Stat` on the config file can be unreliable for some
backends.
2024-06-09 00:05:32 +02:00
Michael Eischer 5e0ea8fcfa pack: move to repository package 2024-05-25 13:13:03 +02:00