amorfo77/restic
1
0
Fork 0
mirror of https://github.com/restic/restic.git synced 2026-07-18 21:24:53 +00:00
Code Issues Projects Releases Wiki Activity
4,532 Commits 9 Branches 53 Tags
190d8e2f510c251cf93b582de1c0cbd842fa4d09
Commit Graph
615 Commits
Author SHA1 Message Date
greatroar 190d8e2f51 Flatten backend.LimitedReadCloser structure 
This inlines the io.LimitedReader into the LimitedReadCloser body to
achieve fewer allocations. Results on linux/amd64:

name                                      old time/op    new time/op    delta
Backend/BenchmarkLoadPartialFile-8           412µs ± 4%     413µs ± 4%    ~     (p=0.634 n=17+17)
Backend/BenchmarkLoadPartialFileOffset-8     455µs ±13%     441µs ±10%    ~     (p=0.072 n=20+18)

name                                      old speed      new speed      delta
Backend/BenchmarkLoadPartialFile-8        10.2GB/s ± 3%  10.2GB/s ± 4%    ~     (p=0.817 n=16+17)
Backend/BenchmarkLoadPartialFileOffset-8  9.25GB/s ±12%  9.54GB/s ± 9%    ~     (p=0.072 n=20+18)

name                                      old alloc/op   new alloc/op   delta
Backend/BenchmarkLoadPartialFile-8            888B ± 0%      872B ± 0%  -1.80%  (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8      888B ± 0%      872B ± 0%  -1.80%  (p=0.000 n=15+15)

name                                      old allocs/op  new allocs/op  delta
Backend/BenchmarkLoadPartialFile-8            18.0 ± 0%      17.0 ± 0%  -5.56%  (p=0.000 n=15+15)
Backend/BenchmarkLoadPartialFileOffset-8      18.0 ± 0%      17.0 ± 0%  -5.56%  (p=0.000 n=15+15)
 2020-06-17 13:11:45 +02:00
greatroar f4cd2a7120 Make backend benchmarks fairer by removing checks 
Checking whether the right data is returned takes up half the time in
some benchmarks. Results for local backend benchmarks on linux/amd64:

name                                      old time/op    new time/op    delta
Backend/BenchmarkLoadFile-8                 4.89ms ± 0%    2.72ms ± 1%   -44.26%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8           936µs ± 6%     439µs ±15%   -53.07%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8     940µs ± 1%     456µs ±10%   -51.50%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                     23.9ms ±14%    24.8ms ±41%      ~     (p=0.690 n=5+5)

name                                      old speed      new speed      delta
Backend/BenchmarkLoadFile-8               3.43GB/s ± 0%  6.16GB/s ± 1%   +79.40%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFile-8        4.48GB/s ± 6%  9.63GB/s ±14%  +114.78%  (p=0.008 n=5+5)
Backend/BenchmarkLoadPartialFileOffset-8  4.46GB/s ± 1%  9.22GB/s ±10%  +106.74%  (p=0.008 n=5+5)
Backend/BenchmarkSave-8                    706MB/s ±13%   698MB/s ±31%      ~     (p=0.690 n=5+5)
 2020-06-17 13:11:45 +02:00
MichaelEischerandGitHub 84475aa3a8 Merge pull request #2730 from greatroar/mount-build-tags 
Simplify build tags for restic mount
 2020-05-23 20:33:43 +02:00
Martin Michlmayr 5cc1760fdf Fix typos 2020-05-16 14:05:26 +08:00
greatroar 649cbec6c5 Simplify build tags for restic mount 
This command can only be built on Darwin, FreeBSD and Linux
(and if we upgrade bazil.org/fuse, only FreeBSD and Linux:
https://github.com/bazil/fuse/issues/224).

Listing the few supported operating systems explicitly here makes
porting restic to new platforms easier.
 2020-05-12 11:30:41 +02:00
greatroar 9f7cd69f13 Move Index.FindBlob to tests 2020-04-29 10:57:01 +02:00
greatroar e7d7b85d59 Merge Repository.{LoadBlob,loadBlob} 
Pushing the allocation logic down into the former loadBlob body means
that fewer allocations have to be performed:

name              old time/op    new time/op    delta
LoadTree-8           478µs ± 1%     481µs ± 2%    ~     (p=0.315 n=9+10)
LoadBlob-8          11.6ms ± 1%    11.6ms ± 2%    ~     (p=0.393 n=10+10)
LoadAndDecrypt-8    13.3ms ± 3%    13.3ms ± 3%    ~     (p=0.905 n=10+9)
LoadIndex-8         33.6ms ± 2%    33.2ms ± 1%  -1.15%  (p=0.028 n=10+9)

name              old alloc/op   new alloc/op   delta
LoadTree-8          41.2kB ± 0%    41.1kB ± 0%  -0.23%  (p=0.000 n=10+10)
LoadBlob-8          2.28kB ± 0%    2.18kB ± 0%  -4.21%  (p=0.000 n=10+10)
LoadAndDecrypt-8    2.10MB ± 0%    2.10MB ± 0%    ~     (all equal)
LoadIndex-8         5.22MB ± 0%    5.22MB ± 0%    ~     (p=0.631 n=10+10)

name              old allocs/op  new allocs/op  delta
LoadTree-8             652 ± 0%       651 ± 0%  -0.15%  (p=0.000 n=10+10)
LoadBlob-8            24.0 ± 0%      23.0 ± 0%  -4.17%  (p=0.000 n=10+10)
LoadAndDecrypt-8      30.0 ± 0%      30.0 ± 0%    ~     (all equal)
LoadIndex-8          30.2k ± 0%     30.2k ± 0%    ~     (p=0.610 n=10+10)

name              old speed      new speed      delta
LoadBlob-8        86.4MB/s ± 1%  85.9MB/s ± 2%    ~     (p=0.393 n=10+10)
LoadAndDecrypt-8  75.4MB/s ± 3%  75.4MB/s ± 3%    ~     (p=0.858 n=10+9)
 2020-04-23 10:04:20 +02:00
greatroar be5a0ff59f Centralize buffer allocation and size checking in Repository.LoadBlob 
Benchmark results for internal/repository:

name              old time/op    new time/op    delta
LoadTree-8           479µs ± 2%     478µs ± 1%   ~     (p=0.780 n=10+9)
LoadBlob-8          11.6ms ± 2%    11.6ms ± 1%   ~     (p=0.631 n=10+10)
LoadAndDecrypt-8    13.2ms ± 2%    13.3ms ± 3%   ~     (p=0.631 n=10+10)

name              old alloc/op   new alloc/op   delta
LoadTree-8          41.2kB ± 0%    41.2kB ± 0%   ~     (all equal)
LoadBlob-8          2.28kB ± 0%    2.28kB ± 0%   ~     (all equal)
LoadAndDecrypt-8    2.10MB ± 0%    2.10MB ± 0%   ~     (all equal)

name              old allocs/op  new allocs/op  delta
LoadTree-8             652 ± 0%       652 ± 0%   ~     (all equal)
LoadBlob-8            24.0 ± 0%      24.0 ± 0%   ~     (all equal)
LoadAndDecrypt-8      30.0 ± 0%      30.0 ± 0%   ~     (all equal)

name              old speed      new speed      delta
LoadBlob-8        86.2MB/s ± 2%  86.4MB/s ± 1%   ~     (p=0.594 n=10+10)
LoadAndDecrypt-8  75.7MB/s ± 2%  75.4MB/s ± 3%   ~     (p=0.617 n=10+10)
 2020-04-23 10:04:20 +02:00
MichaelEischerandGitHub 4f00564574 Merge pull request #2621 from greatroar/fixes 
Some small fixes
 2020-04-18 18:07:08 +02:00
MichaelEischerandGitHub f77477129f Merge pull request #2584 from greatroar/mount-cache-uid-gid 
Cache uid and gid for top directories in internal/fuse
 2020-04-18 17:45:14 +02:00
greatroar 2e31120f89 Remove unused argument to restic.fakeFile 2020-04-18 17:40:13 +02:00
greatroar 8fb2c0d3c1 Typo in crypto test name 2020-04-18 17:39:06 +02:00
greatroar 072cf7b02d Fix debug messages in internal/fuse 2020-04-18 17:39:06 +02:00
greatroar df66daa5c9 Fix context usage in backend tests 
Found by go vet. This is also the only complaint is has.
 2020-04-18 17:39:06 +02:00
MichaelEischerandGitHub 16710454f4 Merge pull request #2628 from MichaelEischer/one-element-pack-lists 
cache: Don't sort one element pack lists
 2020-04-18 17:09:06 +02:00
MichaelEischerandGitHub 7910ff4c0e Merge pull request #2648 from nairb774/iowritestring 
termstatus: Use io.WriteString to output messages.
 2020-04-18 13:48:42 +02:00
MichaelEischerandGitHub c4da9d1e90 Merge pull request #2638 from greatroar/no-close-in-packer 
Don't Close in Packer.Finalize
 2020-04-18 13:07:21 +02:00
MichaelEischerandGitHub a1352906e2 Merge pull request #2622 from greatroar/optimize-packer-manager 
Fix PackerManager benchmark and optimize hashing.Writer
 2020-04-18 12:46:34 +02:00
MichaelEischerandGitHub b7c0d4d8bf Merge pull request #2644 from greatroar/signal-notify-buffered 
Make all signal.Notify channels buffered
 2020-04-18 11:31:02 +02:00
Michael Eischer bdf7ba20cb archiver: Fix race condition triggered by TestArchiverAbortEarlyOnError 
The Save methods of the BlobSaver, FileSaver and TreeSaver return early
on when the archiver is stopped due to an error. For that they select on
both the tomb.Dying() and context.Done() channels, which can lead to a
race condition when the tomb is killed due to an error: The tomb first
closes its Dying channel before canceling all child contexts.
Archiver.SaveDir only aborts its execution once the context was
canceled. When the tomb killing is paused between closing its Dying
channel and canceling the child contexts, this lets the
FileSaver/TreeSaver.Save methods return immediately, however, ScanDir
still reads further files causing the test case to fail.

As a killed tomb always cancels all child contexts and as the Savers
always use a context bound to the tomb, it is sufficient to just use
context.Done() as escape hatch in the Save functions. This fixes the
mismatch between SaveDir and Save.

Adjust the tests to use contexts bound to the tomb for all interactions
with the Savers.
 2020-04-13 18:23:17 +02:00
Brian Atkinson b8da7b1f4d termstatus: Use io.WriteString to output messages. 
The previous implementation was repeating the implementation that is
found inside of io.WriteString. Simplify by making use of the stdlib's
implementation.
 2020-03-26 14:55:00 -07:00
rawtazandGitHub 9efbe98879 Merge pull request #2623 from alrs/internal-restic-err-before-close 
internal/restic: close os.File after checking for error
 2020-03-18 22:23:20 +01:00
rawtazandGitHub 7d9300efca Merge pull request #2637 from greatroar/unused 
Remove Go 1.5 compatibility code
 2020-03-18 22:22:15 +01:00
greatroar 47d4d5bf1b Make all signal.Notify channels buffered 2020-03-12 20:59:39 +01:00
greatroar 74a64c47e4 Move testing logic to test file in internal/pack 2020-03-09 14:32:28 +01:00
greatroar a23e9c86ba Remove closing logic from Packer.Finalize 
The method only ever receives *hashing.Writers, which don't implement
io.Closer. These come from packerManager.findPacker and have their
actual writers closed in Repository.savePacker. Moving the closing logic
to hashing.Writer results in "file already closed" errors.
 2020-03-09 14:31:45 +01:00
greatroar 4de12bf593 Remove restic.RandReader 
math/rand.Rand has implemented Reader since Go 1.6. The repacking tests
are not deterministic, but they weren't before, either.
 2020-03-09 10:00:28 +01:00
greatroar 8cf3bb8737 Revert "Put host last in SSH command line" 
This reverts commit e1969d1e33.
 2020-03-08 16:45:33 +01:00
Michael Eischer b46cc6d57e repository: Don't sort one element pack lists 
When loading a blob, restic first looks up pack files containing the
blob. To avoid unnecessary work an already cached pack file is preferred.
However, if there is only a single pack file to choose from (which is
the normal case) sorting the one-element list won't change anything.
Therefore avoid the unnecessary cache check in that case.
 2020-03-07 10:26:06 +01:00
Lars Lehtonen 4a2156d3f0 internal/restic: close os.File after checking for error 2020-03-05 16:22:46 -08:00
greatroar 41fee11f66 Micro-optimization for hashing.Writer/PackerManager 
name             old time/op    new time/op    delta
PackerManager-8     247ms ± 1%     246ms ± 1%  -0.43%  (p=0.001 n=18+18)

name             old speed      new speed      delta
PackerManager-8   213MB/s ± 1%   214MB/s ± 1%  +0.43%  (p=0.001 n=18+18)

name             old alloc/op   new alloc/op   delta
PackerManager-8    92.2kB ± 0%    91.5kB ± 0%  -0.82%  (p=0.000 n=19+20)

name             old allocs/op  new allocs/op  delta
PackerManager-8     1.43k ± 0%     1.41k ± 0%  -1.67%  (p=0.000 n=20+20)
 2020-03-05 22:30:04 +01:00
greatroar b592614061 Improve PackerManager benchmark 
The previous benchmark spent much of its time allocating RNGs and
generating too many random numbers. It now spends 90% of its time
hashing and half of the rest writing to files.

name             old time/op    new time/op    delta
PackerManager-8     319ms ± 1%     247ms ± 1%  -22.48%  (p=0.000 n=20+18)

name             old speed      new speed      delta
PackerManager-8   143MB/s ± 1%   213MB/s ± 1%  +48.63%  (p=0.000 n=10+18)

name             old alloc/op   new alloc/op   delta
PackerManager-8     635kB ± 0%      92kB ± 0%  -85.48%  (p=0.000 n=10+19)

name             old allocs/op  new allocs/op  delta
PackerManager-8     1.64k ± 0%     1.43k ± 0%  -12.76%  (p=0.000 n=10+20)
 2020-03-05 22:30:03 +01:00
greatroar b7c3039eb2 Remove Go 1.5 compatibility code from PackerManager benchmark 
This alone is enough to speed up the benchmark by ~10%.
 2020-03-05 22:29:06 +01:00
Alexander NeumannandGitHub 52abec967f Merge pull request #2605 from middelink/fix-2604 
Fix running tests on a SELinux enabled system
 2020-03-01 20:25:28 +01:00
Pauline MiddelinkandAlexander Neumann 2828a9c2b0 Fix running tests on a SELinux enabled system 
Archivers TestMetadataChanged incorrectly clears the Extended Attributes
from the expected metadata of the temporary file. This is incorrect as on
SELinux enabled filesystem, as the kernel will automaticly add a SElinux
label. However, since ExtendedAttributes{} != ExtendedAttributes{nil} we
still need to clear them if there are no attributes found.
 2020-03-01 20:23:22 +01:00
Alexander NeumannandGitHub 95da6c1c1d Merge pull request #2589 from greatroar/no-stable-sort 
Replace sort.Stable by sort.Strings
 2020-03-01 19:40:28 +01:00
Alexander NeumannandGitHub 0c03a80fc4 Merge pull request #2592 from greatroar/sftp-ipv6 
Support IPv6 in SFTP backend
 2020-03-01 19:38:44 +01:00
Alexander NeumannandGitHub c50f91b7f9 Merge pull request #2602 from greatroar/no-bufpool 
Remove sync.Pool from internal/repository
 2020-03-01 10:50:57 +01:00
Alexander NeumannandGitHub e851d29565 Merge pull request #2608 from greatroar/simplify-termstatus 
Simplify termstatus
 2020-03-01 10:48:29 +01:00
Alexander NeumannandGitHub b67b7ebfe6 Merge pull request #2583 from greatroar/unused 
Remove some unused or duplicated code
 2020-03-01 10:46:17 +01:00
greatroar 751eba0e68 Remove unnecessary pipe checks in termstatus 
canUpdateStatus has already determined that the file descriptor is not a
pipe.
 2020-02-29 18:03:49 +01:00
greatroar 7447c44484 Use golang.org/x/sys/windows in termstatus 
Some functionality is missing, but at least the types are all defined.
Replaced short, word, dword by their Go names to match the x/sys
convention.
 2020-02-29 18:03:49 +01:00
greatroar c8a672fa29 Remove code copy-pasted from x/crypto/ssh/terminal 2020-02-29 18:03:49 +01:00
greatroar 863ba76494 Drop mattn/go-isatty in favor of crypto/ssh/terminal 2020-02-29 18:03:47 +01:00
greatroar 8526cc6647 Remove sync.Pool from internal/repository 
The pool was used improperly, causing more allocations to be
performed than without it.

name              old time/op    new time/op    delta
SaveAndEncrypt-8    36.8ms ± 2%    36.9ms ± 2%    ~     (p=0.218 n=10+10)

name              old speed      new speed      delta
SaveAndEncrypt-8   114MB/s ± 2%   114MB/s ± 2%    ~     (p=0.218 n=10+10)

name              old alloc/op   new alloc/op   delta
SaveAndEncrypt-8    21.1MB ± 0%    21.0MB ± 0%  -0.44%  (p=0.000 n=10+10)

name              old allocs/op  new allocs/op  delta
SaveAndEncrypt-8      79.0 ± 0%      77.0 ± 0%  -2.53%  (p=0.000 n=10+10)
 2020-02-29 17:54:46 +01:00
greatroar 5cd0bce452 Honor RESTIC_CACHE_DIR on Mac and Windows 2020-02-28 15:44:32 +01:00
rawtazandGitHub 58bd165253 Merge pull request #2581 from aawsome/multiple-hostnames 
Allow multiple hostnames tags
 2020-02-27 08:35:23 +01:00
rawtazandGitHub 65d3fb6b33 Merge pull request #2603 from greatroar/restorer-waitgroup 
Fix unsafe sync.WaitGroup usage in restorer.fileRestorer
 2020-02-27 00:30:59 +01:00
greatroar de5516a90e Fix sync.WaitGroup usage in restorer.fileRestorer 2020-02-27 00:07:49 +01:00
greatroar 4f6fd9fb98 Remove remnant of Go 1.9 compatibility code from tests 2020-02-26 22:23:38 +01:00