In UpgradeRepo, if upgradeRepository fails, the error handler attempts
contingency rollback by re-uploading the original config file.
However, the contingency path unconditionally executed repo.be.Remove(ctx, h)
before calling repo.be.Save. On backends supporting atomic replacement (such
as local filesystems), a failed upgrade (e.g. out-of-disk-space or permissions
error while writing the temporary file) leaves the original, valid version 1
configuration file completely intact on disk.
Unconditionally removing the config file during rollback actively deletes
the working configuration. If the subsequent Save call also fails, the
repository is left permanently corrupted without a config file.
Condition the pre-removal on !repo.be.Properties().HasAtomicReplace, matching
the forward upgrade path in upgradeRepository. Add a unit test asserting that
aborted upgrades on atomic backends preserve the intact original configuration.
---------
Co-authored-by: Michael Eischer <michael.eischer@fau.de>
When loading or iterating a directory tree failed, traverseTreeInner
returned a nil file list together with the error. If the error handler
ignored the error, the caller still called leaveDir with that nil list,
and restore --delete removed every existing entry of the directory in
the target, even though the expected content was simply unknown.
Force disable deletion in that case, the same way it is already done
for invalid node names.
Signed-off-by: Dean Chen <862469039@qq.com>
Motivation: when listing pack files fails during prune (for example due
to a request timeout), the error was returned unwrapped and main printed
it with "%+v", which dumps a stack trace. As discussed in the issue, the
error needs one more level of wrapping before it can be treated as a
fatal error.
Approach: in decidePackAction, wrap an error from listing pack files
with errors.Fatalf("failed to list pack files: %v"). Errors that are
already fatal (e.g. ErrSizeNotMatching) and errors after context
cancellation are returned unchanged. The underlying error is preserved
via Unwrap. Retry log messages are unchanged; only the final error
output is shortened and says where the failure occurred.
Validation: `go test ./internal/repository/ -run Prune` and
`go test ./cmd/restic/ -run Prune` pass. The new test
TestPrunePackListError (backend whose pack list fails) fails without
the change and passes with it. I did not reproduce a real REST backend
timeout.
Report: https://github.com/restic/restic/issues/22120
Signed-off-by: Pujitha Paladugu <10557236+pujitha24@users.noreply.github.com>
Assisted-by: claude-sonnet-5-5 (via Claude Code)
Older repair snapshots versions serialized missing subtrees as nodes:null.
The streaming decoder rejects these trees, preventing prune and other
commands from traversing snapshots that previous releases could read.
Accept null as an empty node list. Add decoder coverage and a regression
that prunes, checks and restores a snapshot with a legacy empty subtree.
Replace select based on context/timer with deterministic pattern.
The tests depend on timing. Thus, use synctest which provides
deterministic clock updates and also speeds up test execution by
skipping sleep times.
- addToPacks now returns uint32 directly and the value is passed through
store() to indexMap.add unchanged, instead of casting between int and
uint32.
- merge() appends idx2.packs to idx.packs without going through
addToPacks, so it now checks the pack count against the uint32 limit
itself + error if exceeded.
- optimize changelog message
Each blob in a repository requires one index entry in memory, which is one
of the largest contributors to restic's memory usage. The indexEntry.packIndex
field, an index into the per-Index packs slice, was stored as a machine-word
int (8 bytes on amd64) although the value is always a small array index.
Store it as a uint32 instead. This shrinks indexEntry from 64 to 56 bytes on
64-bit platforms (56 % 8 == 0, so no padding is added), reducing index memory
by roughly 12% -- about 100 MB for a repository with ~13 million blobs, as
confirmed by profiling. uint32 allows 4.29 billion packs (>30 billion blobs at
the assumed 8 blobs/pack), far beyond any real repository; addToPacks guards
against overflow defensively. The on-disk repository format is unaffected.
Introduce restic.Chunker and ChunkerFactory interfaces with a
repository-backed implementation. The archiver obtains chunkers via
ChunkerFactory() from the repository.
Update the chunker dependency. Move the chunk memory handling into the
fileSaver. This minimizes the logic required in a chunker
implementation.
readNextChunk is a direct derivative of Chunker.Next.