amorfo77/restic
1
0
Fork 0
mirror of https://github.com/restic/restic.git synced 2026-07-17 04:34:53 +00:00
Code Issues Projects Releases Wiki Activity
9,863 Commits 9 Branches 53 Tags
9e007fa3cc937b7e4642bb654476e7e01f868b8a
Commit Graph
585 Commits
Author SHA1 Message Date
Matthias Dötsch 905ca56ee8 Address review feedback on #21937: 
- addToPacks now returns uint32 directly and the value is passed through
  store() to indexMap.add unchanged, instead of casting between int and
  uint32.
- merge() appends idx2.packs to idx.packs without going through
  addToPacks, so it now checks the pack count against the uint32 limit
  itself + error if exceeded.
- optimize changelog message
 2026-07-13 07:25:08 +02:00
Matthias Dötsch e428de5f84 Reduce in-memory index entry size by storing packIndex as uint32 
Each blob in a repository requires one index entry in memory, which is one
of the largest contributors to restic's memory usage. The indexEntry.packIndex
field, an index into the per-Index packs slice, was stored as a machine-word
int (8 bytes on amd64) although the value is always a small array index.

Store it as a uint32 instead. This shrinks indexEntry from 64 to 56 bytes on
64-bit platforms (56 % 8 == 0, so no padding is added), reducing index memory
by roughly 12% -- about 100 MB for a repository with ~13 million blobs, as
confirmed by profiling. uint32 allows 4.29 billion packs (>30 billion blobs at
the assumed 8 blobs/pack), far beyond any real repository; addToPacks guards
against overflow defensively. The on-disk repository format is unaffected.
 2026-07-08 06:01:15 +00:00
Winfried PlappertGitHubMichael Eischer
 		75de8b54e6 restic check and restic repair packs: treat missing packfiles the same as damaged and truncated packfiles (#21845) 
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
 2026-06-28 12:09:42 +02:00
Michael Eischer 8cc69782b5 repository: expose ZeroChunk via chunker factory 2026-06-27 22:23:59 +02:00
Michael Eischer ccb828e87b repository: add ChunkerFactory and decouple archiver from chunker 
Introduce restic.Chunker and ChunkerFactory interfaces with a
repository-backed implementation. The archiver obtains chunkers via
ChunkerFactory() from the repository.
 2026-06-27 22:13:41 +02:00
Gilbert Gilb'sandGitHub db03aed816 feat(backends/s3): add warmup support for check command (#5248) 2026-06-25 20:46:55 +02:00
Michael Eischer 072305c5b3 repository: run version sub-tests of TestAllVersions in parallel 
TestRepositoryIncrementalIndex no longer modified index.Full as this
modified global state and also had no effect on the test result.
 2026-06-20 19:59:37 +02:00
Michael Eischer a1420ea6c7 repository: use separate rand instance per test 
Use separate instances to prevent data races when executing tests in
parallel.
 2026-06-20 19:50:10 +02:00
Michael Eischer b24d05bcb4 skip key decryption in tests 2026-06-20 19:46:22 +02:00
Michael Eischer f91afb2fa8 repository: run TestPruneSmall in parallel to other tests 2026-06-20 19:46:22 +02:00
Michael Eischer 2148508050 repository/checker: speed up test repo creation 
data.TestCreateSnapshot is much faster than archiver.TestSnapshot
 2026-06-20 19:46:22 +02:00
Michael Eischer 37e8f50aeb repository: decrease test data size for streamPack 2026-06-20 19:46:22 +02:00
Michael Eischer 7e5eaf1ead crypto: decrease parameters for calibrate test 2026-06-20 19:46:22 +02:00
Michael Eischer aae0985673 index: decrease depth of test repos 2026-06-20 19:46:22 +02:00
Michael Eischer f9d953d4cd index: test ForAllIndex with generated repo 2026-06-20 19:46:22 +02:00
Michael Eischer 69a3ba5b33 index: speed up oversized test data generation 2026-06-20 19:46:22 +02:00
Michael Eischer 1174e97b30 restic: speed up repository config initialization in tests 
Avoid redundant polynomial generation, which takes a millisecond or so.
 2026-06-20 19:46:22 +02:00
Michael Eischer 440d925467 repository: default to fastest compression in tests 2026-06-20 19:46:22 +02:00
Michael Eischer 9ab5fc59c2 restic: split FileType for backend.FileType 
Equality of constants is enforced via internal/repository/filetype.go
using compile time checks.
 2026-06-20 19:15:40 +02:00
Michael Eischer 6c509f7ac1 Correctly pair backend/restic.FileType with APIs 
Use backend.FileType to interact with the backend. And restic.FileType
to interact with the repository.
 2026-06-20 19:15:40 +02:00
Michael Eischer d9d54a505e restic: move Printer interface from internal/ui/progress 
Move Printer and NewNoopPrinter to internal/restic so repository does
not have to import the ui packages.
 2026-06-20 17:49:20 +02:00
Michael Eischer 0f4236cb39 repository: return unlock func from LockRepo 
Drop the Unlocker interface and return the unlock callback directly
from LockRepo, simplifying callers that only need to defer unlock().
 2026-06-20 17:48:35 +02:00
Michael Eischer 4d1b9cef63 internal/fileio: extract low-level file I/O from internal/fs 
Move TempFile and PreallocateFile into internal/fileio. internal/fs
primarily focuses on converting between data.Node and the actual
filesystem state. Extract the two methods to not pull in unnecessary
dependencies.
 2026-06-20 17:48:35 +02:00
Michael Eischer 1a9decf03e repository: show correct error if zstd decoder cannot be initialized 2026-06-14 16:52:40 +02:00
Michael Eischer 82708ff7f7 prune/repair pack: document ignored errors 2026-06-14 16:51:56 +02:00
Michael EischerandGitHub 7d36449ea8 repository: change LoadBlob, LookupBlob and LookupBlobSize to BlobHandles (#21857) 2026-06-14 15:30:00 +02:00
Michael EischerandGitHub 71f38d266c ensure reliable cleanup of test repository (#21880) 2026-06-14 14:25:00 +02:00
Michael EischerandGitHub 650f658244 repository/index: ensure progress bar cancelation on early index load failure (#21881) 2026-06-14 13:56:24 +02:00
Michael Eischer 406dec29a7 repository: remove redundant mutex in checkForOtherLocks 
forAllLocks already serializes the callback calls. Still has to create a
copy of checkedIDs to prevent a data race between forAllLocks and the
callback.
 2026-06-13 21:37:24 +02:00
Michael Eischer 6fe10a72eb repository: drop mutex from lockHandle 2026-06-13 21:37:06 +02:00
Michael Eischer 1a6d43286a repository: comment cleanups 2026-06-13 21:36:36 +02:00
Michael Eischer 151903d4d4 repository: fix goroutine leak due to delayedCancelContext in tests 2026-06-13 21:35:51 +02:00
Michael Eischer b86568b790 repository: merge Unlocker and lockContext 2026-06-13 21:35:41 +02:00
Michael Eischer a5f3256626 repository: partially deduplicate refresh and refreshStaleLock 2026-06-13 21:33:58 +02:00
Michael Eischer 5989671f87 repository: deduplicate error handling in checkForOtherLocks 2026-06-13 21:33:43 +02:00
Michael Eischer b7f3a1367a repository: add SIGHUP handler only on Unix systems 2026-06-13 21:33:27 +02:00
Michael Eischer 4838b0ae3e repository: cleanup naming of lockCtx 2026-06-13 21:33:19 +02:00
Michael Eischer c87da70af9 repository: separate Lock in-repository from lock handle 
Reduce Lock to a pure data transfer object and move the logic to
lockHandle.
 2026-06-13 21:33:03 +02:00
Michael Eischer 81b6414c55 repository: hide Lock methods 
Unexport lock file methods except String. Lock file operations are only
used within the repository package.
 2026-06-13 21:32:13 +02:00
Michael Eischer 637c1cfb66 repository: unexport internal lock file helpers 
Unexport UnlockCancelDelay, IsInvalidLock, ErrRemovedLock, NewLock,
StaleLockTimeout and ForAllLocks
 2026-06-13 21:31:27 +02:00
Michael Eischer b892b1a150 repository: move lock file handling from restic package 2026-06-13 21:31:23 +02:00
Michael Eischer 3ee465d363 repository: rename Lock function to LockRepo 
Rename `Lock()` to `LockRepo()` to make room for the `Lock` struct.
 2026-06-13 21:24:37 +02:00
Michael EischerandGitHub 8e11f5747d restic: introduce Counter interface to decouple from ui/progress (#21861) 
decouple restic and ui/progress packages
 2026-06-13 21:08:18 +02:00
Michael Eischer c8a0bc2c5e repository: move crypto package to internal/repository/crypto 2026-06-13 20:05:06 +02:00
Michael Eischer b94ef4831d repository: add PackBlob.UncompressedCiphertextLength() 2026-06-13 20:05:05 +02:00
Michael Eischer 1acc19dac5 repository: unexport SearchKey 2026-06-13 19:49:03 +02:00
Michael Eischer 79e92b3bc6 repository: change Unlocker to interface 2026-06-13 19:48:47 +02:00
Michael Eischer 766d9f9f1e repository: unexport openKey 2026-06-13 19:48:31 +02:00
Michael Eischer 131abbb4f0 repository: unexport WarmupJob 2026-06-13 19:48:15 +02:00
Michael Eischer d62e42e620 repository: unexport NewChecker 2026-06-13 19:48:02 +02:00