wip

Fix find, do not skip some snapshots

.github/ISSUE_TEMPLATE.md

@@ -1,14 +1,28 @@
 <!--
-NOTE: Not filling out the issue template needs a good reason, otherwise it may
-take a lot longer to find the problem! Please take the time to help us
-debugging the problem by collecting information, even if it seems irrelevant to
-you. Thanks!
 
-If you have a question, the forum at https://discourse.restic.net is a better place.
-Please do not create issues for usage or documentation questions! We're using
-the GitHub issue tracker mainly for tracking bugs and feature requests.
+Welcome! - We kindly ask that you:
+
+  1. Fill out the issue template below - not doing so needs a good reason.
+  2. Use the forum if you have a question rather than a bug or feature request.
+
+The forum is at: https://forum.restic.net
+
+NOTE: Not filling out the issue template needs a good reason, as otherwise it
+may take a lot longer to find the problem, not to mention it can take up a lot
+more time which can otherwise be spent on development. Please also take the
+time to help us debug the issue by collecting relevant information, even if
+it doesn't seem to be relevant to you. Thanks!
+
+The forum is a better place for questions about restic or general suggestions
+and topics, e.g. usage or documentation questions! This issue tracker is mainly
+for tracking bugs and feature requests directly relating to the development of
+the software itself, rather than the project.
+
+Thanks for understanding, and for contributing to the project!
+
 -->
 
+
 ## Output of `restic version`
 
 
@@ -24,10 +38,10 @@ This section should include at least:
    information to diagnose the problem!
 -->
 
-
 ## What backend/server/service did you use to store the repository?
 
 
+
 ## Expected behavior
 
 <!--
@@ -48,12 +62,14 @@ The more time you spend describing an easy way to reproduce the behavior (if
 this is possible), the easier it is for the project developers to fix it!
 -->
 
-
 ## Do you have any idea what may have caused this?
 
 
+
 ## Do you have an idea how to solve the issue?
 
+
+
 ## Did restic help you or made you happy in any way?
 
 <!--

.github/PULL_REQUEST_TEMPLATE.md

@@ -25,7 +25,7 @@ Link issues and relevant forum posts here.
 - [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
 - [ ] I have added tests for all changes in this PR
 - [ ] I have added documentation for the changes (in the manual)
-- [ ] There's an entry in the `CHANGELOG.md` file that describe the changes for our users
+- [ ] There's a new file in `changelog/unreleased/` that describes the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/TEMPLATE))
 - [ ] I have run `gofmt` on the code in all commits
 - [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
 - [ ] I'm done, this Pull Request is ready for review

.travis.yml

@@ -1,30 +1,20 @@
 language: go
 sudo: false
 
-go:
-  - 1.8.x
-  - 1.9.x
-
-os:
-  - linux
-  - osx
-
-env:
-  matrix:
-    RESTIC_TEST_FUSE=0
-
 matrix:
-  exclude:
-    - os: osx
-      go: 1.8.x
-    - os: linux
-      go: 1.9.x
   include:
     - os: linux
-      go: 1.9.x
+      go: "1.9.x"
+      env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0 RESTIC_BUILD_SOLARIS=0
+
+    # only run fuse and cloud backends tests on Travis for the latest Go on Linux
+    - os: linux
+      go: "1.10.x"
       sudo: true
-      env:
-        RESTIC_TEST_FUSE=1
+
+    - os: osx
+      go: "1.10.x"
+      env: RESTIC_TEST_FUSE=0 RESTIC_TEST_CLOUD_BACKENDS=0
 
 branches:
   only:

CHANGELOG.md

@@ -1,4 +1,584 @@
-Changelog for restic 0.8.1 (UNRELEASED)
+Changelog for restic 0.9.1 (2018-06-10)
+=======================================
+
+The following sections list the changes in restic 0.9.1 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1801: Add limiting bandwidth to the rclone backend
+ * Fix #1822: Allow uploading large files to MS Azure
+ * Fix #1825: Correct `find` to not skip snapshots
+ * Fix #1833: Fix caching files on error
+ * Fix #1834: Resolve deadlock
+
+Details
+-------
+
+ * Bugfix #1801: Add limiting bandwidth to the rclone backend
+
+   The rclone backend did not respect `--limit-upload` or `--limit-download`. Oftentimes it's
+   not necessary to use this, as the limiting in rclone itself should be used because it gives much
+   better results, but in case a remote instance of rclone is used (e.g. called via ssh), it is still
+   relevant to limit the bandwidth from restic to rclone.
+
+   https://github.com/restic/restic/issues/1801
+
+ * Bugfix #1822: Allow uploading large files to MS Azure
+
+   Sometimes, restic creates files to be uploaded to the repository which are quite large, e.g.
+   when saving directories with many entries or very large files. The MS Azure API does not allow
+   uploading files larger that 256MiB directly, rather restic needs to upload them in blocks of
+   100MiB. This is now implemented.
+
+   https://github.com/restic/restic/issues/1822
+
+ * Bugfix #1825: Correct `find` to not skip snapshots
+
+   Under certain circumstances, the `find` command was found to skip snapshots containing
+   directories with files to look for when the directories haven't been modified at all, and were
+   already printed as part of a different snapshot. This is now corrected.
+
+   In addition, we've switched to our own matching/pattern implementation, so now things like
+   `restic find "/home/user/foo/**/main.go"` are possible.
+
+   https://github.com/restic/restic/issues/1825
+   https://github.com/restic/restic/issues/1823
+
+ * Bugfix #1833: Fix caching files on error
+
+   During `check` it may happen that different threads access the same file in the backend, which
+   is then downloaded into the cache only once. When that fails, only the thread which is
+   responsible for downloading the file signals the correct error. The other threads just assume
+   that the file has been downloaded successfully and then get an error when they try to access the
+   cached file.
+
+   https://github.com/restic/restic/issues/1833
+
+ * Bugfix #1834: Resolve deadlock
+
+   When the "scanning" process restic runs to find out how much data there is does not finish before
+   the backup itself is done, restic stops doing anything. This is resolved now.
+
+   https://github.com/restic/restic/issues/1834
+   https://github.com/restic/restic/pull/1835
+
+
+Changelog for restic 0.9.0 (2018-05-21)
+=======================================
+
+The following sections list the changes in restic 0.9.0 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1608: Respect time stamp for new backup when reading from stdin
+ * Fix #1652: Ignore/remove invalid lock files
+ * Fix #1730: Ignore sockets for restore
+ * Fix #1684: Fix backend tests for rest-server
+ * Fix #1745: Correctly parse the argument to --tls-client-cert
+ * Enh #1433: Support UTF-16 encoding and process Byte Order Mark
+ * Enh #1561: Allow using rclone to access other services
+ * Enh #1665: Improve cache handling for `restic check`
+ * Enh #1721: Add `cache` command to list cache dirs
+ * Enh #1758: Allow saving OneDrive folders in Windows
+ * Enh #549: Rework archiver code
+ * Enh #1552: Use Google Application Default credentials
+ * Enh #1477: Accept AWS_SESSION_TOKEN for the s3 backend
+ * Enh #1648: Ignore AWS permission denied error when creating a repository
+ * Enh #1649: Add illumos/Solaris support
+ * Enh #1709: Improve messages `restic check` prints
+ * Enh #827: Add --new-password-file flag for non-interactive password changes
+ * Enh #1735: Allow keeping a time range of snaphots
+ * Enh #1782: Use default AWS credentials chain for S3 backend
+
+Details
+-------
+
+ * Bugfix #1608: Respect time stamp for new backup when reading from stdin
+
+   When reading backups from stdin (via `restic backup --stdin`), restic now uses the time stamp
+   for the new backup passed in `--time`.
+
+   https://github.com/restic/restic/issues/1608
+   https://github.com/restic/restic/pull/1703
+
+ * Bugfix #1652: Ignore/remove invalid lock files
+
+   This corrects a bug introduced recently: When an invalid lock file in the repo is encountered
+   (e.g. if the file is empty), the code used to ignore that, but now returns the error. Now, invalid
+   files are ignored for the normal lock check, and removed when `restic unlock --remove-all` is
+   run.
+
+   https://github.com/restic/restic/issues/1652
+   https://github.com/restic/restic/pull/1653
+
+ * Bugfix #1730: Ignore sockets for restore
+
+   We've received a report and correct the behavior in which the restore code aborted restoring a
+   directory when a socket was encountered. Unix domain socket files cannot be restored (they are
+   created on the fly once a process starts listening). The error handling was corrected, and in
+   addition we're now ignoring sockets during restore.
+
+   https://github.com/restic/restic/issues/1730
+   https://github.com/restic/restic/pull/1731
+
+ * Bugfix #1684: Fix backend tests for rest-server
+
+   The REST server for restic now requires an explicit parameter (`--no-auth`) if no
+   authentication should be allowed. This is fixed in the tests.
+
+   https://github.com/restic/restic/pull/1684
+
+ * Bugfix #1745: Correctly parse the argument to --tls-client-cert
+
+   Previously, the --tls-client-cert method attempt to read ARGV[1] (hardcoded) instead of the
+   argument that was passed to it. This has been corrected.
+
+   https://github.com/restic/restic/issues/1745
+   https://github.com/restic/restic/pull/1746
+
+ * Enhancement #1433: Support UTF-16 encoding and process Byte Order Mark
+
+   On Windows, text editors commonly leave a Byte Order Mark at the beginning of the file to define
+   which encoding is used (oftentimes UTF-16). We've added code to support processing the BOMs in
+   text files, like the exclude files, the password file and the file passed via `--files-from`.
+   This does not apply to any file being saved in a backup, those are not touched and archived as they
+   are.
+
+   https://github.com/restic/restic/issues/1433
+   https://github.com/restic/restic/issues/1738
+   https://github.com/restic/restic/pull/1748
+
+ * Enhancement #1561: Allow using rclone to access other services
+
+   We've added the ability to use rclone to store backup data on all backends that it supports. This
+   was done in collaboration with Nick, the author of rclone. You can now use it to first configure a
+   service, then restic manages the rest (starting and stopping rclone). For details, please see
+   the manual.
+
+   https://github.com/restic/restic/issues/1561
+   https://github.com/restic/restic/pull/1657
+   https://rclone.org
+
+ * Enhancement #1665: Improve cache handling for `restic check`
+
+   For safety reasons, restic does not use a local metadata cache for the `restic check` command,
+   so that data is loaded from the repository and restic can check it's in good condition. When the
+   cache is disabled, restic will fetch each tiny blob needed for checking the integrity using a
+   separate backend request. For non-local backends, that will take a long time, and depending on
+   the backend (e.g. B2) may also be much more expensive.
+
+   This PR adds a few commits which will change the behavior as follows:
+
+   * When `restic check` is called without any additional parameters, it will build a new cache in a
+   temporary directory, which is removed at the end of the check. This way, we'll get readahead for
+   metadata files (so restic will fetch the whole file when the first blob from the file is
+   requested), but all data is freshly fetched from the storage backend. This is the default
+   behavior and will work for almost all users.
+
+   * When `restic check` is called with `--with-cache`, the default on-disc cache is used. This
+   behavior hasn't changed since the cache was introduced.
+
+   * When `--no-cache` is specified, restic falls back to the old behavior, and read all tiny blobs
+   in separate requests.
+
+   https://github.com/restic/restic/issues/1665
+   https://github.com/restic/restic/issues/1694
+   https://github.com/restic/restic/pull/1696
+
+ * Enhancement #1721: Add `cache` command to list cache dirs
+
+   The command `cache` was added, it allows listing restic's cache directoriers together with
+   the last usage. It also allows removing old cache dirs without having to access a repo, via
+   `restic cache --cleanup`
+
+   https://github.com/restic/restic/issues/1721
+   https://github.com/restic/restic/pull/1749
+
+ * Enhancement #1758: Allow saving OneDrive folders in Windows
+
+   Restic now contains a bugfix to two libraries, which allows saving OneDrive folders in
+   Windows. In order to use the newer versions of the libraries, the minimal version required to
+   compile restic is now Go 1.9.
+
+   https://github.com/restic/restic/issues/1758
+   https://github.com/restic/restic/pull/1765
+
+ * Enhancement #549: Rework archiver code
+
+   The core archiver code and the complementary code for the `backup` command was rewritten
+   completely. This resolves very annoying issues such as 549. The first backup with this release
+   of restic will likely result in all files being re-read locally, so it will take a lot longer. The
+   next backup after that will be fast again.
+
+   Basically, with the old code, restic took the last path component of each to-be-saved file or
+   directory as the top-level file/directory within the snapshot. This meant that when called as
+   `restic backup /home/user/foo`, the snapshot would contain the files in the directory
+   `/home/user/foo` as `/foo`.
+
+   This is not the case any more with the new archiver code. Now, restic works very similar to what
+   `tar` does: When restic is called with an absolute path to save, then it'll preserve the
+   directory structure within the snapshot. For the example above, the snapshot would contain
+   the files in the directory within `/home/user/foo` in the snapshot. For relative
+   directories, it only preserves the relative path components. So `restic backup user/foo`
+   will save the files as `/user/foo` in the snapshot.
+
+   While we were at it, the status display and notification system was completely rewritten. By
+   default, restic now shows which files are currently read (unless `--quiet` is specified) in a
+   multi-line status display.
+
+   The `backup` command also gained a new option: `--verbose`. It can be specified once (which
+   prints a bit more detail what restic is doing) or twice (which prints a line for each
+   file/directory restic encountered, together with some statistics).
+
+   Another issue that was resolved is the new code only reads two files at most. The old code would
+   read way too many files in parallel, thereby slowing down the backup process on spinning discs a
+   lot.
+
+   https://github.com/restic/restic/issues/549
+   https://github.com/restic/restic/issues/1286
+   https://github.com/restic/restic/issues/446
+   https://github.com/restic/restic/issues/1344
+   https://github.com/restic/restic/issues/1416
+   https://github.com/restic/restic/issues/1456
+   https://github.com/restic/restic/issues/1145
+   https://github.com/restic/restic/issues/1160
+   https://github.com/restic/restic/pull/1494
+
+ * Enhancement #1552: Use Google Application Default credentials
+
+   Google provide libraries to generate appropriate credentials with various fallback
+   sources. This change uses the library to generate our GCS client, which allows us to make use of
+   these extra methods.
+
+   This should be backward compatible with previous restic behaviour while adding the
+   additional capabilities to auth from Google's internal metadata endpoints. For users
+   running restic in GCP this can make authentication far easier than it was before.
+
+   https://github.com/restic/restic/pull/1552
+   https://developers.google.com/identity/protocols/application-default-credentials
+
+ * Enhancement #1477: Accept AWS_SESSION_TOKEN for the s3 backend
+
+   Before, it was not possible to use s3 backend with AWS temporary security credentials(with
+   AWS_SESSION_TOKEN). This change gives higher priority to credentials.EnvAWS credentials
+   provider.
+
+   https://github.com/restic/restic/issues/1477
+   https://github.com/restic/restic/pull/1479
+   https://github.com/restic/restic/pull/1647
+
+ * Enhancement #1648: Ignore AWS permission denied error when creating a repository
+
+   It's not possible to use s3 backend scoped to a subdirectory(with specific permissions).
+   Restic doesn't try to create repository in a subdirectory, when 'bucket exists' of parent
+   directory check fails due to permission issues.
+
+   https://github.com/restic/restic/pull/1648
+
+ * Enhancement #1649: Add illumos/Solaris support
+
+   https://github.com/restic/restic/pull/1649
+
+ * Enhancement #1709: Improve messages `restic check` prints
+
+   Some messages `restic check` prints are not really errors, so from now on restic does not treat
+   them as errors any more and exits cleanly.
+
+   https://github.com/restic/restic/pull/1709
+   https://forum.restic.net/t/what-is-the-standard-procedure-to-follow-if-a-backup-or-restore-is-interrupted/571/2
+
+ * Enhancement #827: Add --new-password-file flag for non-interactive password changes
+
+   This makes it possible to change a repository password without being prompted.
+
+   https://github.com/restic/restic/issues/827
+   https://github.com/restic/restic/pull/1720
+   https://forum.restic.net/t/changing-repo-password-without-prompt/591
+
+ * Enhancement #1735: Allow keeping a time range of snaphots
+
+   We've added the `--keep-within` option to the `forget` command. It instructs restic to keep
+   all snapshots within the given duration since the newest snapshot. For example, running
+   `restic forget --keep-within 5m7d` will keep all snapshots which have been made in the five
+   months and seven days since the latest snapshot.
+
+   https://github.com/restic/restic/pull/1735
+
+ * Enhancement #1782: Use default AWS credentials chain for S3 backend
+
+   Adds support for file credentials to the S3 backend (e.g. ~/.aws/credentials), and reorders
+   the credentials chain for the S3 backend to match AWS's standard, which is static credentials,
+   env vars, credentials file, and finally remote.
+
+   https://github.com/restic/restic/pull/1782
+
+
+Changelog for restic 0.8.3 (2018-02-26)
+=======================================
+
+The following sections list the changes in restic 0.8.3 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1633: Fixed unexpected 'pack file cannot be listed' error
+ * Fix #1641: Ignore files with invalid names in the repo
+ * Fix #1638: Handle errors listing files in the backend
+ * Enh #1497: Add --read-data-subset flag to check command
+ * Enh #1560: Retry all repository file download errors
+ * Enh #1623: Don't check for presence of files in the backend before writing
+ * Enh #1634: Upgrade B2 client library, reduce HTTP requests
+
+Details
+-------
+
+ * Bugfix #1633: Fixed unexpected 'pack file cannot be listed' error
+
+   Due to a regression introduced in 0.8.2, the `rebuild-index` and `prune` commands failed to
+   read pack files with size of 587, 588, 589 or 590 bytes.
+
+   https://github.com/restic/restic/issues/1633
+   https://github.com/restic/restic/pull/1635
+
+ * Bugfix #1641: Ignore files with invalid names in the repo
+
+   The release 0.8.2 introduced a bug: when restic encounters files in the repo which do not have a
+   valid name, it tries to load a file with a name of lots of zeroes instead of ignoring it. This is now
+   resolved, invalid file names are just ignored.
+
+   https://github.com/restic/restic/issues/1641
+   https://github.com/restic/restic/pull/1643
+   https://forum.restic.net/t/help-fixing-repo-no-such-file/485/3
+
+ * Bugfix #1638: Handle errors listing files in the backend
+
+   A user reported in the forum that restic completes a backup although a concurrent `prune`
+   operation was running. A few error messages were printed, but the backup was attempted and
+   completed successfully. No error code was returned.
+
+   This should not happen: The repository is exclusively locked during `prune`, so when `restic
+   backup` is run in parallel, it should abort and return an error code instead.
+
+   It was found that the bug was in the code introduced only recently, which retries a List()
+   operation on the backend should that fail. It is now corrected.
+
+   https://github.com/restic/restic/pull/1638
+   https://forum.restic.net/t/restic-backup-returns-0-exit-code-when-already-locked/484
+
+ * Enhancement #1497: Add --read-data-subset flag to check command
+
+   This change introduces ability to check integrity of a subset of repository data packs. This
+   can be used to spread integrity check of larger repositories over a period of time.
+
+   https://github.com/restic/restic/issues/1497
+   https://github.com/restic/restic/pull/1556
+
+ * Enhancement #1560: Retry all repository file download errors
+
+   Restic will now retry failed downloads, similar to other operations.
+
+   https://github.com/restic/restic/pull/1560
+
+ * Enhancement #1623: Don't check for presence of files in the backend before writing
+
+   Before, all backend implementations were required to return an error if the file that is to be
+   written already exists in the backend. For most backends, that means making a request (e.g. via
+   HTTP) and returning an error when the file already exists.
+
+   This is not accurate, the file could have been created between the HTTP request testing for it,
+   and when writing starts, so we've relaxed this requeriment, which saves one additional HTTP
+   request per newly added file.
+
+   https://github.com/restic/restic/pull/1623
+
+ * Enhancement #1634: Upgrade B2 client library, reduce HTTP requests
+
+   We've upgraded the B2 client library restic uses to access BackBlaze B2. This reduces the
+   number of HTTP requests needed to upload a new file from two to one, which should improve
+   throughput to B2.
+
+   https://github.com/restic/restic/pull/1634
+
+
+Changelog for restic 0.8.2 (2018-02-17)
+=======================================
+
+The following sections list the changes in restic 0.8.2 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
+ * Fix #1512: Restore directory permissions as the last step
+ * Fix #1528: Correctly create missing subdirs in data/
+ * Fix #1590: Strip spaces for lines read via --files-from
+ * Fix #1589: Complete intermediate index upload
+ * Fix #1594: Google Cloud Storage: Use generic HTTP transport
+ * Fix #1595: Backup: Remove bandwidth display
+ * Enh #1522: Add support for TLS client certificate authentication
+ * Enh #1541: Reduce number of remote requests during repository check
+ * Enh #1567: Reduce number of backend requests for rebuild-index and prune
+ * Enh #1507: Only reload snapshots once per minute for fuse mount
+ * Enh #1538: Reduce memory allocations for querying the index
+ * Enh #1549: Speed up querying across indices and scanning existing files
+ * Enh #1554: Fuse/mount: Correctly handle EOF, add template option
+ * Enh #1564: Don't terminate ssh on SIGINT
+ * Enh #1579: Retry Backend.List() in case of errors
+ * Enh #1584: Limit index file size
+
+Details
+-------
+
+ * Bugfix #1506: Limit bandwith at the http.RoundTripper for HTTP based backends
+
+   https://github.com/restic/restic/issues/1506
+   https://github.com/restic/restic/pull/1511
+
+ * Bugfix #1512: Restore directory permissions as the last step
+
+   This change allows restoring into directories that were not writable during backup. Before,
+   restic created the directory, set the read-only mode and then failed to create files in the
+   directory. This change now restores the directory (with its permissions) as the very last
+   step.
+
+   https://github.com/restic/restic/issues/1512
+   https://github.com/restic/restic/pull/1536
+
+ * Bugfix #1528: Correctly create missing subdirs in data/
+
+   https://github.com/restic/restic/issues/1528
+   https://github.com/restic/restic/pull/1529
+
+ * Bugfix #1590: Strip spaces for lines read via --files-from
+
+   Leading and trailing spaces in lines read via `--files-from` are now stripped, so it behaves
+   the same as with lines read via `--exclude-file`.
+
+   https://github.com/restic/restic/issues/1590
+   https://github.com/restic/restic/pull/1613
+
+ * Bugfix #1589: Complete intermediate index upload
+
+   After a user posted a comprehensive report of what he observed, we were able to find a bug and
+   correct it: During backup, restic uploads so-called "intermediate" index files. When the
+   backup finishes during a transfer of such an intermediate index, the upload is cancelled, but
+   the backup is finished without an error. This leads to an inconsistent state, where the
+   snapshot references data that is contained in the repo, but is not referenced in any index.
+
+   The situation can be resolved by building a new index with `rebuild-index`, but looks very
+   confusing at first. Since all the data got uploaded to the repo successfully, there was no risk
+   of data loss, just minor inconvenience for our users.
+
+   https://github.com/restic/restic/pull/1589
+   https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406
+
+ * Bugfix #1594: Google Cloud Storage: Use generic HTTP transport
+
+   It was discovered that the Google Cloud Storage backend did not use the generic HTTP transport,
+   so things such as bandwidth limiting with `--limit-upload` did not work. This is resolved now.
+
+   https://github.com/restic/restic/pull/1594
+
+ * Bugfix #1595: Backup: Remove bandwidth display
+
+   This commit removes the bandwidth displayed during backup process. It is misleading and
+   seldomly correct, because it's neither the "read bandwidth" (only for the very first backup)
+   nor the "upload bandwidth". Many users are confused about (and rightly so), c.f. #1581, #1033,
+   #1591
+
+   We'll eventually replace this display with something more relevant when the new archiver code
+   is ready.
+
+   https://github.com/restic/restic/pull/1595
+
+ * Enhancement #1522: Add support for TLS client certificate authentication
+
+   Support has been added for using a TLS client certificate for authentication to HTTP based
+   backend. A file containing the PEM encoded private key and certificate can be set using the
+   `--tls-client-cert` option.
+
+   https://github.com/restic/restic/issues/1522
+   https://github.com/restic/restic/pull/1524
+
+ * Enhancement #1541: Reduce number of remote requests during repository check
+
+   This change eliminates redundant remote repository calls and significantly improves
+   repository check time.
+
+   https://github.com/restic/restic/issues/1541
+   https://github.com/restic/restic/pull/1548
+
+ * Enhancement #1567: Reduce number of backend requests for rebuild-index and prune
+
+   We've found a way to reduce then number of backend requests for the `rebuild-index` and `prune`
+   operations. This significantly speeds up the operations for high-latency backends.
+
+   https://github.com/restic/restic/issues/1567
+   https://github.com/restic/restic/pull/1574
+   https://github.com/restic/restic/pull/1575
+
+ * Enhancement #1507: Only reload snapshots once per minute for fuse mount
+
+   https://github.com/restic/restic/pull/1507
+
+ * Enhancement #1538: Reduce memory allocations for querying the index
+
+   This change reduces the internal memory allocations when the index data structures in memory
+   are queried if a blob (part of a file) already exists in the repo. It should speed up backup a bit,
+   and maybe even reduce RAM usage.
+
+   https://github.com/restic/restic/pull/1538
+
+ * Enhancement #1549: Speed up querying across indices and scanning existing files
+
+   This change increases the whenever a blob (part of a file) is searched for in a restic
+   repository. This will reduce cpu usage some when backing up files already backed up by restic.
+   Cpu usage is further decreased when scanning files.
+
+   https://github.com/restic/restic/pull/1549
+
+ * Enhancement #1554: Fuse/mount: Correctly handle EOF, add template option
+
+   We've added the `--snapshot-template` string, which can be used to specify a template for a
+   snapshot directory. In addition, accessing data after the end of a file via the fuse mount is now
+   handled correctly.
+
+   https://github.com/restic/restic/pull/1554
+
+ * Enhancement #1564: Don't terminate ssh on SIGINT
+
+   We've reworked the code which runs the `ssh` login for the sftp backend so that it can prompt for a
+   password (if needed) but does not exit when the user presses CTRL+C (SIGINT) e.g. during
+   backup. This allows restic to properly shut down when it receives SIGINT and remove the lock
+   file from the repo, afterwards exiting the `ssh` process.
+
+   https://github.com/restic/restic/pull/1564
+   https://github.com/restic/restic/pull/1588
+
+ * Enhancement #1579: Retry Backend.List() in case of errors
+
+   https://github.com/restic/restic/pull/1579
+
+ * Enhancement #1584: Limit index file size
+
+   Before, restic would create a single new index file on `prune` or `rebuild-index`, this may
+   lead to memory problems when this huge index is created and loaded again. We're now limiting the
+   size of the index file, and split newly created index files into several smaller ones. This
+   allows restic to be more memory-efficient.
+
+   https://github.com/restic/restic/issues/1412
+   https://github.com/restic/restic/issues/979
+   https://github.com/restic/restic/issues/526
+   https://github.com/restic/restic/pull/1584
+
+
+Changelog for restic 0.8.1 (2017-12-27)
 =======================================
 
 The following sections list the changes in restic 0.8.1 relevant to
@@ -9,7 +589,7 @@ Summary
 
  * Fix #1457: Improve s3 backend with DigitalOcean Spaces
  * Fix #1454: Correct cache dir location for Windows and Darwin
- * Fix #1457: Disable handling SIGPIPE
+ * Fix #1459: Disable handling SIGPIPE
  * Chg #1452: Do not save atime by default
  * Enh #1436: Add code to detect old cache directories
  * Enh #1439: Improve cancellation logic
@@ -30,7 +610,7 @@ Details
 
    https://github.com/restic/restic/pull/1454
 
- * Bugfix #1457: Disable handling SIGPIPE
+ * Bugfix #1459: Disable handling SIGPIPE
 
    We've disabled handling SIGPIPE again. Turns out, writing to broken TCP connections also
    raised SIGPIPE, so restic exits on the first write to a broken connection. Instead, restic
@@ -87,7 +667,7 @@ Summary
  * Fix #1256: Re-enable workaround for S3 backend
  * Fix #1291: Reuse backend TCP connections to BackBlaze B2
  * Fix #1317: Run prune when `forget --prune` is called with just snapshot IDs
- * Fix #1292: Remove implicit path `/restic` for the s3 backend
+ * Fix #1437: Remove implicit path `/restic` for the s3 backend
  * Enh #1102: Add subdirectory `ids` to fuse mount
  * Enh #1114: Add `--cacert` to specify TLS certificates to check against
  * Enh #1216: Add upload/download limiting
@@ -96,11 +676,11 @@ Summary
  * Enh #1367: Allow comments in files read from via `--file-from`
  * Enh #448: Sftp backend prompts for password
  * Enh #510: Add `dump` command
- * Enh #29: Add local metadata cache
+ * Enh #1040: Add local metadata cache
  * Enh #1249: Add `latest` symlink in fuse mount
  * Enh #1269: Add `--compact` to `forget` command
  * Enh #1281: Google Cloud Storage backend needs less permissions
- * Enh #1303: Make `check` print `no errors found` explicitly
+ * Enh #1319: Make `check` print `no errors found` explicitly
  * Enh #1353: Retry failed backend requests
 
 Details
@@ -148,7 +728,7 @@ Details
 
    https://github.com/restic/restic/pull/1317
 
- * Bugfix #1292: Remove implicit path `/restic` for the s3 backend
+ * Bugfix #1437: Remove implicit path `/restic` for the s3 backend
 
    The s3 backend used the subdir `restic` within a bucket if no explicit path after the bucket name
    was specified. Since this version, restic does not use this default path any more. If you
@@ -226,7 +806,7 @@ Details
    https://github.com/restic/restic/issues/510
    https://github.com/restic/restic/pull/1346
 
- * Enhancement #29: Add local metadata cache
+ * Enhancement #1040: Add local metadata cache
 
    We've added a local cache for metadata so that restic doesn't need to load all metadata
    (snapshots, indexes, ...) from the repo each time it starts. By default the cache is active, but
@@ -270,7 +850,7 @@ Details
 
    https://github.com/restic/restic/pull/1281
 
- * Enhancement #1303: Make `check` print `no errors found` explicitly
+ * Enhancement #1319: Make `check` print `no errors found` explicitly
 
    The `check` command now explicetly prints `No errors were found` when no errors could be found.
 
@@ -326,9 +906,9 @@ Summary
  * Enh #1044: Improve `restore`, do not traverse/load excluded directories
  * Enh #1061: Add Dockerfile and official Docker image
  * Enh #1126: Use the standard Go git repository layout, use `dep` for vendoring
- * Enh #211: Add support for storing backups on Google Cloud Storage
- * Enh #1144: Properly report errors when reading files with exclude patterns.
- * Enh #609: Add support for storing backups on Microsoft Azure Blob Storage
+ * Enh #1134: Add support for storing backups on Google Cloud Storage
+ * Enh #1144: Properly report errors when reading files with exclude patterns
+ * Enh #1149: Add support for storing backups on Microsoft Azure Blob Storage
  * Enh #1196: Add `--group-by` to `forget` command for flexible grouping
  * Enh #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
  * Enh #1205: Allow specifying time/date for a backup with `--time`
@@ -409,17 +989,17 @@ Details
 
    https://github.com/restic/restic/pull/1126
 
- * Enhancement #211: Add support for storing backups on Google Cloud Storage
+ * Enhancement #1134: Add support for storing backups on Google Cloud Storage
 
    https://github.com/restic/restic/issues/211
    https://github.com/restic/restic/pull/1134
    https://github.com/restic/restic/pull/1052
 
- * Enhancement #1144: Properly report errors when reading files with exclude patterns.
+ * Enhancement #1144: Properly report errors when reading files with exclude patterns
 
    https://github.com/restic/restic/pull/1144
 
- * Enhancement #609: Add support for storing backups on Microsoft Azure Blob Storage
+ * Enhancement #1149: Add support for storing backups on Microsoft Azure Blob Storage
 
    The library we're using to access the service requires Go 1.8, so restic now needs at least Go
    1.8.
@@ -435,7 +1015,7 @@ Details
  * Enhancement #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
 
    https://github.com/restic/restic/pull/1203
-   https://github.com/restic/restic/pull/1082
+   https://github.com/restic/restic/pull/1082#issuecomment-326279920
 
  * Enhancement #1205: Allow specifying time/date for a backup with `--time`
 
@@ -475,12 +1055,12 @@ Details
  * Enhancement #1055: Create subdirs below `data/` for local/sftp backends
 
    The local and sftp backends now create the subdirs below `data/` on open/init. This way, restic
-   makes sure that they always exist. This is connected to an issue for the sftp server:
+   makes sure that they always exist. This is connected to an issue for the sftp server.
 
    https://github.com/restic/restic/issues/1055
+   https://github.com/restic/rest-server/pull/11#issuecomment-309879710
    https://github.com/restic/restic/pull/1077
    https://github.com/restic/restic/pull/1105
-   https://github.com/restic/rest-server/pull/11#issuecomment-309879710
 
  * Enhancement #1067: Allow loading credentials for s3 from IAM
 
@@ -492,7 +1072,7 @@ Details
 
  * Enhancement #1073: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
 
-   The `migrate` command for chaning the `s3legacy` layout to the `default` layout for s3
+   The `migrate` command for changing the `s3legacy` layout to the `default` layout for s3
    backends has been improved: It can now be restarted with `restic migrate --force s3_layout`
    and automatically retries operations on error.
 
@@ -655,14 +1235,14 @@ restic users. The changes are ordered by importance.
 Summary
 -------
 
- * Enh #953: Make `forget` consistent
- * Enh #965: Unify repository layout for all backends
+ * Enh #957: Make `forget` consistent
+ * Enh #966: Unify repository layout for all backends
  * Enh #962: Improve memory and runtime for the s3 backend
 
 Details
 -------
 
- * Enhancement #953: Make `forget` consistent
+ * Enhancement #957: Make `forget` consistent
 
    The `forget` command was corrected to be more consistent in which snapshots are to be
    forgotten. It is possible that the new code removes more snapshots than before, so please
@@ -671,7 +1251,7 @@ Details
    https://github.com/restic/restic/issues/953
    https://github.com/restic/restic/pull/957
 
- * Enhancement #965: Unify repository layout for all backends
+ * Enhancement #966: Unify repository layout for all backends
 
    Up to now the s3 backend used a special repository layout. We've decided to unify the repository
    layout and implemented the default layout also for the s3 backend. For creating a new

CONTRIBUTING.md

@@ -120,7 +120,8 @@ down to the following steps:
 
  6. You will receive comments on your code and the feature or bug that they
     address. Maybe you need to rework some minor things, in this case push new
-    commits to the branch you created for the pull request, they will be
+    commits to the branch you created for the pull request (or amend the
+    existing commit, use common sense to decide which is better), they will be
     automatically added to the pull request.
 
  7. If your pull request changes anything that users should be aware of (a

GOVERNANCE.md

@@ -0,0 +1,27 @@
+# restic project governance
+
+## Overview
+
+The restic project uses a governance model commonly described as Benevolent
+Dictator For Life (BDFL). This document outlines our understanding of what this
+means. It is derived from the [i3 window manager project
+governance](https://raw.githubusercontent.com/i3/i3/next/.github/GOVERNANCE.md).
+
+## Roles
+
+* user: anyone who interacts with the restic project
+* core contributor: a handful of people who have contributed significantly to
+  the project by any means (issue triage, support, documentation, code, etc.).
+  Core contributors are recognizable via GitHub’s "Member" badge.
+* Benevolent Dictator For Life (BDFL): a single individual who makes decisions
+  when consensus cannot be reached. restic's current BDFL is [@fd0](https://github.com/fd0).
+
+## Decision making process
+
+In general, we try to reach consensus in discussions. In case consensus cannot
+be reached, the BDFL makes a decision.
+
+## Contribution process
+
+The contribution process is described in a separate document called
+[CONTRIBUTING](CONTRIBUTING.md).

Gopkg.lock

@@ -10,38 +10,38 @@
 [[projects]]
   name = "cloud.google.com/go"
   packages = ["compute/metadata"]
-  revision = "2d3a6656c17a60b0815b7e06ab0be04eacb6e613"
-  version = "v0.16.0"
+  revision = "4b98a6370e36d7a85192e7bad08a4ebd82eac2a8"
+  version = "v0.20.0"
 
 [[projects]]
   name = "github.com/Azure/azure-sdk-for-go"
-  packages = ["storage"]
-  revision = "7692b0cef22674113fcf71cc17ac3ccc1a7fef48"
-  version = "v11.2.2-beta"
+  packages = ["storage","version"]
+  revision = "56332fec5b308fbb6615fa1af6117394cdba186d"
+  version = "v15.0.0"
 
 [[projects]]
   name = "github.com/Azure/go-autorest"
   packages = ["autorest","autorest/adal","autorest/azure","autorest/date"]
-  revision = "c67b24a8e30d876542a85022ebbdecf0e5a935e8"
-  version = "v9.4.1"
+  revision = "ed4b7f5bf1ec0c9ede1fda2681d96771282f2862"
+  version = "v10.4.0"
 
 [[projects]]
   name = "github.com/cenkalti/backoff"
   packages = ["."]
-  revision = "61153c768f31ee5f130071d08fc82b85208528de"
-  version = "v1.1.0"
+  revision = "2ea60e5f094469f9e65adb9cd103795b73ae743e"
+  version = "v2.0.0"
 
 [[projects]]
   name = "github.com/cpuguy83/go-md2man"
   packages = ["md2man"]
-  revision = "1d903dcb749992f3741d744c0f8376b4bd7eb3e1"
-  version = "v1.0.7"
+  revision = "20f5889cbdc3c73dbd2862796665e7c465ade7d1"
+  version = "v1.0.8"
 
 [[projects]]
   name = "github.com/dgrijalva/jwt-go"
   packages = ["."]
-  revision = "dbeaa9332f19a944acb5736b4456cfcc02140e29"
-  version = "v3.1.0"
+  revision = "06ea1031745cb8b3dab3f6a236daf2b0aa468b7e"
+  version = "v3.2.0"
 
 [[projects]]
   branch = "master"
@@ -52,20 +52,26 @@
 [[projects]]
   name = "github.com/elithrar/simple-scrypt"
   packages = ["."]
-  revision = "2325946f714c95de4a6088202c402fbdfa64163b"
-  version = "v1.2.0"
+  revision = "d150773194090feb6c897805a7bcea8d49544e2c"
+  version = "v1.3.0"
 
 [[projects]]
   name = "github.com/go-ini/ini"
   packages = ["."]
-  revision = "32e4c1e6bc4e7d0d8451aa6b75200d19e37a536a"
-  version = "v1.32.0"
+  revision = "6333e38ac20b8949a8dd68baa3650f4dee8f39f0"
+  version = "v1.33.0"
 
 [[projects]]
-  branch = "master"
   name = "github.com/golang/protobuf"
   packages = ["proto"]
-  revision = "1e59b77b52bf8e4b449a57e6f79f21226d571845"
+  revision = "925541529c1fa6821df4e44ce2723319eb2be768"
+  version = "v1.0.0"
+
+[[projects]]
+  name = "github.com/google/go-cmp"
+  packages = ["cmp","cmp/internal/diff","cmp/internal/function","cmp/internal/value"]
+  revision = "8099a9787ce5dc5984ed879a3bda47dc730a8e97"
+  version = "v0.1.0"
 
 [[projects]]
   name = "github.com/inconshreveable/mousetrap"
@@ -74,10 +80,10 @@
   version = "v1.0"
 
 [[projects]]
-  branch = "master"
   name = "github.com/juju/ratelimit"
   packages = ["."]
   revision = "59fac5042749a5afb9af70e813da1dd5474f0167"
+  version = "1.0.1"
 
 [[projects]]
   branch = "master"
@@ -87,27 +93,39 @@
 
 [[projects]]
   name = "github.com/kurin/blazer"
-  packages = ["b2","base","internal/b2types","internal/blog"]
-  revision = "e269a1a17bb6aec278c06a57cb7e8f8d0d333e04"
-  version = "v0.2.1"
+  packages = ["b2","base","internal/b2assets","internal/b2types","internal/blog","x/window"]
+  revision = "318e9768bf9a0fe52a64b9f8fe74f4f5caef6452"
+  version = "v0.4.4"
 
 [[projects]]
-  branch = "master"
-  name = "github.com/minio/go-homedir"
+  name = "github.com/marstr/guid"
   packages = ["."]
-  revision = "21304a94172ae3a09dee2cd86a12fb6f842138c7"
+  revision = "8bd9a64bf37eb297b492a4101fb28e80ac0b290f"
+  version = "v1.1.0"
+
+[[projects]]
+  name = "github.com/mattn/go-isatty"
+  packages = ["."]
+  revision = "0360b2af4f38e8d38c7fce2a9f4e702702d73a39"
+  version = "v0.0.3"
 
 [[projects]]
   name = "github.com/minio/minio-go"
   packages = [".","pkg/credentials","pkg/encrypt","pkg/policy","pkg/s3signer","pkg/s3utils","pkg/set"]
-  revision = "57a8ae886b49af6eb0d2c27c2d007ed2f71e1da5"
-  version = "4.0.3"
+  revision = "66252c2a3c15f7b90cc8493d497a04ac3b6e3606"
+  version = "5.0.0"
+
+[[projects]]
+  branch = "master"
+  name = "github.com/mitchellh/go-homedir"
+  packages = ["."]
+  revision = "b8bc1bf767474819792c23f32d8286a45736f1c6"
 
 [[projects]]
   branch = "master"
   name = "github.com/ncw/swift"
   packages = ["."]
-  revision = "c95c6e5c2d1a3d37fc44c8c6dc9e231c7500667d"
+  revision = "b2a7479cf26fa841ff90dd932d0221cb5c50782d"
 
 [[projects]]
   name = "github.com/pkg/errors"
@@ -124,14 +142,14 @@
 [[projects]]
   name = "github.com/pkg/sftp"
   packages = ["."]
-  revision = "98203f5a8333288eb3163b7c667d4260fe1333e9"
-  version = "1.0.0"
+  revision = "49488377fa2f14143ba3067cf7555f60f6c7b550"
+  version = "1.5.0"
 
 [[projects]]
   name = "github.com/pkg/xattr"
   packages = ["."]
-  revision = "23c75e3f6c1d8b13b3dd905b011a7f38a06044b7"
-  version = "v0.2.1"
+  revision = "1d7b7ffe7c46974a836eb583b7452f22de1c18cf"
+  version = "v0.2.3"
 
 [[projects]]
   name = "github.com/restic/chunker"
@@ -142,26 +160,26 @@
 [[projects]]
   name = "github.com/russross/blackfriday"
   packages = ["."]
-  revision = "4048872b16cc0fc2c5fd9eacf0ed2c2fedaa0c8c"
-  version = "v1.5"
+  revision = "55d61fa8aa702f59229e6cff85793c22e580eaf5"
+  version = "v1.5.1"
 
 [[projects]]
-  name = "github.com/satori/uuid"
+  name = "github.com/satori/go.uuid"
   packages = ["."]
-  revision = "879c5887cd475cd7864858769793b2ceb0d44feb"
-  version = "v1.1.0"
+  revision = "f58768cc1a7a7e77a3bd49e98cdd21419399b6a3"
+  version = "v1.2.0"
 
 [[projects]]
   name = "github.com/sirupsen/logrus"
   packages = ["."]
-  revision = "f006c2ac4710855cf0f916dd6b77acf6b048dc6e"
-  version = "v1.0.3"
+  revision = "c155da19408a8799da419ed3eeb0cb5db0ad5dbc"
+  version = "v1.0.5"
 
 [[projects]]
   name = "github.com/spf13/cobra"
   packages = [".","doc"]
-  revision = "7b2c5ac9fc04fc5efafb60700713d4fa609b777b"
-  version = "v0.0.1"
+  revision = "a1f051bc3eba734da4772d60e2d677f47cf93ef4"
+  version = "v0.0.2"
 
 [[projects]]
   name = "github.com/spf13/pflag"
@@ -172,32 +190,44 @@
 [[projects]]
   branch = "master"
   name = "golang.org/x/crypto"
-  packages = ["curve25519","ed25519","ed25519/internal/edwards25519","pbkdf2","poly1305","scrypt","ssh","ssh/terminal"]
-  revision = "94eea52f7b742c7cbe0b03b22f0c4c8631ece122"
+  packages = ["argon2","blake2b","curve25519","ed25519","ed25519/internal/edwards25519","internal/chacha20","pbkdf2","poly1305","scrypt","ssh","ssh/terminal"]
+  revision = "4ec37c66abab2c7e02ae775328b2ff001c3f025a"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/net"
-  packages = ["context","context/ctxhttp"]
-  revision = "a8b9294777976932365dabb6640cf1468d95c70f"
+  packages = ["context","context/ctxhttp","http2","http2/hpack","idna","lex/httplex"]
+  revision = "6078986fec03a1dcc236c34816c71b0e05018fda"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/oauth2"
   packages = [".","google","internal","jws","jwt"]
-  revision = "f95fa95eaa936d9d87489b15d1d18b97c1ba9c28"
+  revision = "fdc9e635145ae97e6c2cb777c48305600cf515cb"
+
+[[projects]]
+  branch = "master"
+  name = "golang.org/x/sync"
+  packages = ["errgroup"]
+  revision = "1d60e4601c6fd243af51cc01ddf169918a5407ca"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/sys"
-  packages = ["unix","windows"]
-  revision = "8b4580aae2a0dd0c231a45d3ccb8434ff533b840"
+  packages = ["cpu","unix","windows"]
+  revision = "7db1c3b1a98089d0071c84f646ff5c96aad43682"
+
+[[projects]]
+  name = "golang.org/x/text"
+  packages = ["collate","collate/build","encoding","encoding/internal","encoding/internal/identifier","encoding/unicode","internal/colltab","internal/gen","internal/tag","internal/triegen","internal/ucd","internal/utf8internal","language","runes","secure/bidirule","transform","unicode/bidi","unicode/cldr","unicode/norm","unicode/rangetable"]
+  revision = "f21a4dfb5e38f5895301dc265a8def02365cc3d0"
+  version = "v0.3.0"
 
 [[projects]]
   branch = "master"
   name = "google.golang.org/api"
   packages = ["gensupport","googleapi","googleapi/internal/uritemplates","storage/v1"]
-  revision = "3a1d936b7575b82197a1fea0632218dd07b1e65c"
+  revision = "dbbc13f71100fa6ece308335445fca6bb0dd5c2f"
 
 [[projects]]
   name = "google.golang.org/appengine"
@@ -207,13 +237,19 @@
 
 [[projects]]
   branch = "v2"
+  name = "gopkg.in/tomb.v2"
+  packages = ["."]
+  revision = "d5d1b5820637886def9eef33e03a27a9f166942c"
+
+[[projects]]
   name = "gopkg.in/yaml.v2"
   packages = ["."]
-  revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"
+  revision = "5420a8b6744d3b0345ab293f6fcba19c978f1183"
+  version = "v2.2.1"
 
 [solve-meta]
   analyzer-name = "dep"
   analyzer-version = 1
-  inputs-digest = "f0a207197cb502238ac87ca8e07b2640c02ec380a50b036e09ef87e40e31ca2d"
+  inputs-digest = "a5de339cba7570216b212439b90e1e6c384c94be8342fe7755b7cb66aa0a3440"
   solver-name = "gps-cdcl"
   solver-version = 1

README.rst

@@ -29,7 +29,7 @@ and add some data:
 
 .. code-block:: console
 
-    $ restic -r /tmp/backup backup ~/work
+    $ restic --repo /tmp/backup backup ~/work
     enter password for repository:
     scan [/home/user/work]
     scanned 764 directories, 1816 files in 0:00
@@ -57,6 +57,7 @@ Therefore, restic supports the following backends for storing backups natively:
 - `BackBlaze B2 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2>`__
 - `Microsoft Azure Blob Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#microsoft-azure-blob-storage>`__
 - `Google Cloud Storage <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#google-cloud-storage>`__
+- And many other services via the `rclone <https://rclone.org>`__ `Backend <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#other-services-via-rclone>`__
 
 Design Principles
 -----------------
@@ -102,7 +103,7 @@ News
 ----
 
 You can follow the restic project on Twitter `@resticbackup <https://twitter.com/resticbackup>`__ or by subscribing to
-the `development blog <https://restic.github.io/blog/>`__.
+the `development blog <https://restic.net/blog/>`__.
 
 License
 -------
@@ -112,8 +113,8 @@ complete text in ``LICENSE``.
 
 .. |Documentation| image:: https://readthedocs.org/projects/restic/badge/?version=latest
    :target: https://restic.readthedocs.io/en/latest/?badge=latest
-.. |Build Status| image:: https://travis-ci.org/restic/restic.svg?branch=master
-   :target: https://travis-ci.org/restic/restic
+.. |Build Status| image:: https://travis-ci.com/restic/restic.svg?branch=master
+   :target: https://travis-ci.com/restic/restic
 .. |Build status| image:: https://ci.appveyor.com/api/projects/status/nuy4lfbgfbytw92q/branch/master?svg=true
    :target: https://ci.appveyor.com/project/fd0/restic/branch/master
 .. |Report Card| image:: https://goreportcard.com/badge/github.com/restic/restic

VERSION

@@ -1 +1 @@
-0.8.1
+0.9.1

appveyor.yml

@@ -17,8 +17,8 @@ init:
 
 install:
   - rmdir c:\go /s /q
-  - appveyor DownloadFile https://storage.googleapis.com/golang/go1.9.windows-amd64.msi
-  - msiexec /i go1.9.windows-amd64.msi /q
+  - appveyor DownloadFile https://dl.google.com/go/go1.10.windows-amd64.msi
+  - msiexec /i go1.10.windows-amd64.msi /q
   - go version
   - go env
   - appveyor DownloadFile http://sourceforge.netcologne.de/project/gnuwin32/tar/1.13-1/tar-1.13-1-bin.zip -FileName tar.zip

build.go

@@ -1,4 +1,33 @@
-// +build ignore
+// BSD 2-Clause License
+//
+// Copyright (c) 2016-2018, Alexander Neumann <alexander@bumpern.de>
+// All rights reserved.
+//
+// This file has been copied from the repository at:
+// https://github.com/fd0/build-go
+//
+// Redistribution and use in source and binary forms, with or without
+// modification, are permitted provided that the following conditions are met:
+//
+// * Redistributions of source code must retain the above copyright notice, this
+//   list of conditions and the following disclaimer.
+//
+// * Redistributions in binary form must reproduce the above copyright notice,
+//   this list of conditions and the following disclaimer in the documentation
+//   and/or other materials provided with the distribution.
+//
+// THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+// AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+// IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+// DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+// FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+// DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+// SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+// CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+// OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+// OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+// +build ignore_build_go
 
 package main
 
@@ -11,30 +40,39 @@ import (
 	"path"
 	"path/filepath"
 	"runtime"
+	"strconv"
 	"strings"
 )
 
+// config contains the configuration for the program to build.
+var config = Config{
+	Name:      "restic",                              // name of the program executable and directory
+	Namespace: "github.com/restic/restic",            // subdir of GOPATH, e.g. "github.com/foo/bar"
+	Main:      "github.com/restic/restic/cmd/restic", // package name for the main package
+	Tests: []string{ // tests to run
+		"github.com/restic/restic/internal/...",
+		"github.com/restic/restic/cmd/...",
+	},
+	MinVersion: GoVersion{Major: 1, Minor: 9, Patch: 0}, // minimum Go version supported
+}
+
+// Config configures the build.
+type Config struct {
+	Name       string
+	Namespace  string
+	Main       string
+	Tests      []string
+	MinVersion GoVersion
+}
+
 var (
 	verbose    bool
 	keepGopath bool
 	runTests   bool
 	enableCGO  bool
+	enablePIE  bool
 )
 
-var config = struct {
-	Name      string
-	Namespace string
-	Main      string
-	Tests     []string
-}{
-	Name:      "restic",                              // name of the program executable and directory
-	Namespace: "github.com/restic/restic",            // subdir of GOPATH, e.g. "github.com/foo/bar"
-	Main:      "github.com/restic/restic/cmd/restic", // package name for the main package
-	Tests: []string{ // tests to run
-		"github.com/restic/restic/internal/...",
-		"github.com/restic/restic/cmd/..."},
-}
-
 // specialDir returns true if the file begins with a special character ('.' or '_').
 func specialDir(name string) bool {
 	if name == "." {
@@ -137,7 +175,6 @@ func copyFile(dst, src string) error {
 	if err != nil {
 		return err
 	}
-	defer fsrc.Close()
 
 	if err = os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
 		fmt.Printf("MkdirAll(%v)\n", filepath.Dir(dst))
@@ -148,17 +185,28 @@ func copyFile(dst, src string) error {
 	if err != nil {
 		return err
 	}
-	defer fdst.Close()
 
-	_, err = io.Copy(fdst, fsrc)
+	if _, err = io.Copy(fdst, fsrc); err != nil {
+		return err
+	}
+
+	if err == nil {
+		err = fsrc.Close()
+	}
+
+	if err == nil {
+		err = fdst.Close()
+	}
+
 	if err == nil {
 		err = os.Chmod(dst, fi.Mode())
 	}
+
 	if err == nil {
 		err = os.Chtimes(dst, fi.ModTime(), fi.ModTime())
 	}
 
-	return err
+	return nil
 }
 
 // die prints the message with fmt.Fprintf() to stderr and exits with an error
@@ -178,8 +226,11 @@ func showUsage(output io.Writer) {
 	fmt.Fprintf(output, "  -T     --test          run tests\n")
 	fmt.Fprintf(output, "  -o     --output        set output file name\n")
 	fmt.Fprintf(output, "         --enable-cgo    use CGO to link against libc\n")
+	fmt.Fprintf(output, "         --enable-pie    use PIE buildmode\n")
 	fmt.Fprintf(output, "         --goos value    set GOOS for cross-compilation\n")
 	fmt.Fprintf(output, "         --goarch value  set GOARCH for cross-compilation\n")
+	fmt.Fprintf(output, "         --goarm value   set GOARM for cross-compilation\n")
+	fmt.Fprintf(output, "         --tempdir dir   use a specific directory for compilation\n")
 }
 
 func verbosePrintf(message string, args ...interface{}) {
@@ -205,13 +256,28 @@ func cleanEnv() (env []string) {
 }
 
 // build runs "go build args..." with GOPATH set to gopath.
-func build(cwd, goos, goarch, gopath string, args ...string) error {
+func build(cwd string, ver GoVersion, goos, goarch, goarm, gopath string, args ...string) error {
 	a := []string{"build"}
-	a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
-	a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
+
+	if ver.AtLeast(GoVersion{1, 10, 0}) {
+		verbosePrintf("Go version is at least 1.10, using new syntax for -gcflags\n")
+		// use new prefix
+		a = append(a, "-asmflags", fmt.Sprintf("all=-trimpath=%s", gopath))
+		a = append(a, "-gcflags", fmt.Sprintf("all=-trimpath=%s", gopath))
+	} else {
+		a = append(a, "-asmflags", fmt.Sprintf("-trimpath=%s", gopath))
+		a = append(a, "-gcflags", fmt.Sprintf("-trimpath=%s", gopath))
+	}
+	if enablePIE {
+		a = append(a, "-buildmode=pie")
+	}
+
 	a = append(a, args...)
 	cmd := exec.Command("go", a...)
 	cmd.Env = append(cleanEnv(), "GOPATH="+gopath, "GOARCH="+goarch, "GOOS="+goos)
+	if goarm != "" {
+		cmd.Env = append(cmd.Env, "GOARM="+goarm)
+	}
 	if !enableCGO {
 		cmd.Env = append(cmd.Env, "CGO_ENABLED=0")
 	}
@@ -219,7 +285,7 @@ func build(cwd, goos, goarch, gopath string, args ...string) error {
 	cmd.Dir = cwd
 	cmd.Stdout = os.Stdout
 	cmd.Stderr = os.Stderr
-	verbosePrintf("go %s\n", args)
+	verbosePrintf("go %s\n", a)
 
 	return cmd.Run()
 }
@@ -300,10 +366,89 @@ func (cs Constants) LDFlags() string {
 	return strings.Join(l, " ")
 }
 
+// GoVersion is the version of Go used to compile the project.
+type GoVersion struct {
+	Major int
+	Minor int
+	Patch int
+}
+
+// ParseGoVersion parses the Go version s. If s cannot be parsed, the returned GoVersion is null.
+func ParseGoVersion(s string) (v GoVersion) {
+	if !strings.HasPrefix(s, "go") {
+		return
+	}
+
+	s = s[2:]
+	data := strings.Split(s, ".")
+	if len(data) < 2 || len(data) > 3 {
+		// invalid version
+		return GoVersion{}
+	}
+
+	var err error
+
+	v.Major, err = strconv.Atoi(data[0])
+	if err != nil {
+		return GoVersion{}
+	}
+
+	// try to parse the minor version while removing an eventual suffix (like
+	// "rc2" or so)
+	for s := data[1]; s != ""; s = s[:len(s)-1] {
+		v.Minor, err = strconv.Atoi(s)
+		if err == nil {
+			break
+		}
+	}
+
+	if v.Minor == 0 {
+		// no minor version found
+		return GoVersion{}
+	}
+
+	if len(data) >= 3 {
+		v.Patch, err = strconv.Atoi(data[2])
+		if err != nil {
+			return GoVersion{}
+		}
+	}
+
+	return
+}
+
+// AtLeast returns true if v is at least as new as other. If v is empty, true is returned.
+func (v GoVersion) AtLeast(other GoVersion) bool {
+	var empty GoVersion
+
+	// the empty version satisfies all versions
+	if v == empty {
+		return true
+	}
+
+	if v.Major < other.Major {
+		return false
+	}
+
+	if v.Minor < other.Minor {
+		return false
+	}
+
+	if v.Patch < other.Patch {
+		return false
+	}
+
+	return true
+}
+
+func (v GoVersion) String() string {
+	return fmt.Sprintf("Go %d.%d.%d", v.Major, v.Minor, v.Patch)
+}
+
 func main() {
-	ver := runtime.Version()
-	if strings.HasPrefix(ver, "go1") && ver < "go1.8" {
-		fmt.Fprintf(os.Stderr, "Go version %s detected, restic requires at least Go 1.8\n", ver)
+	ver := ParseGoVersion(runtime.Version())
+	if !ver.AtLeast(config.MinVersion) {
+		fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", ver, config.MinVersion)
 		os.Exit(1)
 	}
 
@@ -314,6 +459,9 @@ func main() {
 
 	targetGOOS := runtime.GOOS
 	targetGOARCH := runtime.GOARCH
+	targetGOARM := ""
+
+	gopath := ""
 
 	var outputFilename string
 
@@ -337,16 +485,24 @@ func main() {
 		case "-o", "--output":
 			skipNext = true
 			outputFilename = params[i+1]
+		case "--tempdir":
+			skipNext = true
+			gopath = params[i+1]
 		case "-T", "--test":
 			runTests = true
 		case "--enable-cgo":
 			enableCGO = true
+		case "--enable-pie":
+			enablePIE = true
 		case "--goos":
 			skipNext = true
 			targetGOOS = params[i+1]
 		case "--goarch":
 			skipNext = true
 			targetGOARCH = params[i+1]
+		case "--goarm":
+			skipNext = true
+			targetGOARM = params[i+1]
 		case "-h":
 			showUsage(os.Stdout)
 			return
@@ -357,6 +513,8 @@ func main() {
 		}
 	}
 
+	verbosePrintf("detected Go version %v\n", ver)
+
 	if len(buildTags) == 0 {
 		verbosePrintf("adding build-tag release\n")
 		buildTags = []string{"release"}
@@ -373,9 +531,11 @@ func main() {
 		die("Getwd(): %v\n", err)
 	}
 
-	gopath, err := ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
-	if err != nil {
-		die("TempDir(): %v\n", err)
+	if gopath == "" {
+		gopath, err = ioutil.TempDir("", fmt.Sprintf("%v-build-", config.Name))
+		if err != nil {
+			die("TempDir(): %v\n", err)
+		}
 	}
 
 	verbosePrintf("create GOPATH at %v\n", gopath)
@@ -431,7 +591,7 @@ func main() {
 		"-o", output, config.Main,
 	}
 
-	err = build(filepath.Join(gopath, "src"), targetGOOS, targetGOARCH, gopath, args...)
+	err = build(filepath.Join(gopath, "src"), ver, targetGOOS, targetGOARCH, targetGOARM, gopath, args...)
 	if err != nil {
 		die("build failed: %v\n", err)
 	}

changelog/0.7.1_2017-07-22/issue-1055

@@ -2,9 +2,9 @@ Enhancement: Create subdirs below `data/` for local/sftp backends
 
 The local and sftp backends now create the subdirs below `data/` on
 open/init. This way, restic makes sure that they always exist. This is
-connected to an issue for the sftp server:
+connected to an issue for the sftp server.
 
-https://github.com/restic/rest-server/pull/11#issuecomment-309879710
 https://github.com/restic/restic/issues/1055
+https://github.com/restic/rest-server/pull/11#issuecomment-309879710
 https://github.com/restic/restic/pull/1077
 https://github.com/restic/restic/pull/1105

changelog/0.7.1_2017-07-22/issue-1073

@@ -1,6 +1,6 @@
 Enhancement: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
 
-The `migrate` command for chaning the `s3legacy` layout to the `default` layout
+The `migrate` command for changing the `s3legacy` layout to the `default` layout
 for s3 backends has been improved: It can now be restarted with `restic migrate
 --force s3_layout` and automatically retries operations on error.
 

changelog/0.7.2_2017-09-13/pull-1144

@@ -1,3 +1,3 @@
-Enhancement: Properly report errors when reading files with exclude patterns.
+Enhancement: Properly report errors when reading files with exclude patterns
 
 https://github.com/restic/restic/pull/1144

changelog/0.8.1_2017-12-27/issue-1457

@@ -1,4 +1,4 @@
 Bugfix: Improve s3 backend with DigitalOcean Spaces
 
-https://github.com/restic/restic/pull/1459
 https://github.com/restic/restic/issues/1457
+https://github.com/restic/restic/pull/1459

changelog/0.8.2_2018-02-17/issue-1506

@@ -0,0 +1,4 @@
+Bugfix: Limit bandwith at the http.RoundTripper for HTTP based backends
+
+https://github.com/restic/restic/issues/1506
+https://github.com/restic/restic/pull/1511

changelog/0.8.2_2018-02-17/issue-1512

@@ -0,0 +1,9 @@
+Bugfix: Restore directory permissions as the last step
+
+This change allows restoring into directories that were not writable during
+backup. Before, restic created the directory, set the read-only mode and then
+failed to create files in the directory. This change now restores the directory
+(with its permissions) as the very last step.
+
+https://github.com/restic/restic/issues/1512
+https://github.com/restic/restic/pull/1536

changelog/0.8.2_2018-02-17/issue-1522

@@ -0,0 +1,8 @@
+Enhancement: Add support for TLS client certificate authentication
+
+Support has been added for using a TLS client certificate for authentication to
+HTTP based backend. A file containing the PEM encoded private key and
+certificate can be set using the `--tls-client-cert` option.
+
+https://github.com/restic/restic/issues/1522
+https://github.com/restic/restic/pull/1524

changelog/0.8.2_2018-02-17/issue-1528

@@ -0,0 +1,4 @@
+Bugfix: Correctly create missing subdirs in data/
+
+https://github.com/restic/restic/issues/1528
+https://github.com/restic/restic/pull/1529

changelog/0.8.2_2018-02-17/issue-1541

@@ -0,0 +1,7 @@
+Enhancement: Reduce number of remote requests during repository check
+
+This change eliminates redundant remote repository calls and significantly
+improves repository check time.
+
+https://github.com/restic/restic/issues/1541
+https://github.com/restic/restic/pull/1548

changelog/0.8.2_2018-02-17/issue-1567

@@ -0,0 +1,9 @@
+Enhancement: Reduce number of backend requests for rebuild-index and prune
+
+We've found a way to reduce then number of backend requests for the
+`rebuild-index` and `prune` operations. This significantly speeds up the
+operations for high-latency backends.
+
+https://github.com/restic/restic/issues/1567
+https://github.com/restic/restic/pull/1574
+https://github.com/restic/restic/pull/1575

changelog/0.8.2_2018-02-17/issue-1590

@@ -0,0 +1,7 @@
+Bugfix: Strip spaces for lines read via --files-from
+
+Leading and trailing spaces in lines read via `--files-from` are now stripped,
+so it behaves the same as with lines read via `--exclude-file`.
+
+https://github.com/restic/restic/issues/1590
+https://github.com/restic/restic/pull/1613

changelog/0.8.2_2018-02-17/pull-1507

@@ -0,0 +1,3 @@
+Enhancement: Only reload snapshots once per minute for fuse mount
+
+https://github.com/restic/restic/pull/1507

changelog/0.8.2_2018-02-17/pull-1538

@@ -0,0 +1,7 @@
+Enhancement: Reduce memory allocations for querying the index
+
+This change reduces the internal memory allocations when the index data
+structures in memory are queried if a blob (part of a file) already exists in
+the repo. It should speed up backup a bit, and maybe even reduce RAM usage.
+
+https://github.com/restic/restic/pull/1538

changelog/0.8.2_2018-02-17/pull-1549

@@ -0,0 +1,7 @@
+Enhancement: Speed up querying across indices and scanning existing files
+
+This change increases the whenever a blob (part of a file) is searched for in a
+restic repository.  This will reduce cpu usage some when backing up files already
+backed up by restic.  Cpu usage is further decreased when scanning files.
+
+https://github.com/restic/restic/pull/1549

changelog/0.8.2_2018-02-17/pull-1554

@@ -0,0 +1,7 @@
+Enhancement: fuse/mount: Correctly handle EOF, add template option
+
+We've added the `--snapshot-template` string, which can be used to specify a
+template for a snapshot directory. In addition, accessing data after the end of
+a file via the fuse mount is now handled correctly.
+
+https://github.com/restic/restic/pull/1554

changelog/0.8.2_2018-02-17/pull-1564

@@ -0,0 +1,10 @@
+Enhancement: Don't terminate ssh on SIGINT
+
+We've reworked the code which runs the `ssh` login for the sftp backend so that
+it can prompt for a password (if needed) but does not exit when the user
+presses CTRL+C (SIGINT) e.g. during backup. This allows restic to properly shut
+down when it receives SIGINT and remove the lock file from the repo, afterwards
+exiting the `ssh` process.
+
+https://github.com/restic/restic/pull/1564
+https://github.com/restic/restic/pull/1588

changelog/0.8.2_2018-02-17/pull-1579

@@ -0,0 +1,3 @@
+Enhancement: Retry Backend.List() in case of errors
+
+https://github.com/restic/restic/pull/1579

changelog/0.8.2_2018-02-17/pull-1584

@@ -0,0 +1,12 @@
+Enhancement: Limit index file size
+
+Before, restic would create a single new index file on `prune` or
+`rebuild-index`, this may lead to memory problems when this huge index is
+created and loaded again. We're now limiting the size of the index file, and
+split newly created index files into several smaller ones. This allows restic
+to be more memory-efficient.
+
+https://github.com/restic/restic/pull/1584
+https://github.com/restic/restic/issues/1412
+https://github.com/restic/restic/issues/979
+https://github.com/restic/restic/issues/526

changelog/0.8.2_2018-02-17/pull-1589

@@ -0,0 +1,17 @@
+Bugfix: Complete intermediate index upload
+
+After a user posted a comprehensive report of what he observed, we were able to
+find a bug and correct it: During backup, restic uploads so-called
+"intermediate" index files. When the backup finishes during a transfer of such
+an intermediate index, the upload is cancelled, but the backup is finished
+without an error. This leads to an inconsistent state, where the snapshot
+references data that is contained in the repo, but is not referenced in any
+index.
+
+The situation can be resolved by building a new index with `rebuild-index`, but
+looks very confusing at first. Since all the data got uploaded to the repo
+successfully, there was no risk of data loss, just minor inconvenience for our
+users.
+
+https://github.com/restic/restic/pull/1589
+https://forum.restic.net/t/error-loading-tree-check-prune-and-forget-gives-error-b2-backend/406

changelog/0.8.2_2018-02-17/pull-1594

@@ -0,0 +1,7 @@
+Bugfix: Google Cloud Storage: Use generic HTTP transport
+
+It was discovered that the Google Cloud Storage backend did not use the generic
+HTTP transport, so things such as bandwidth limiting with `--limit-upload` did
+not work. This is resolved now.
+
+https://github.com/restic/restic/pull/1594

changelog/0.8.2_2018-02-17/pull-1595

@@ -0,0 +1,11 @@
+Bugfix: backup: Remove bandwidth display
+
+This commit removes the bandwidth displayed during backup process. It is
+misleading and seldomly correct, because it's neither the "read
+bandwidth" (only for the very first backup) nor the "upload bandwidth".
+Many users are confused about (and rightly so), c.f. #1581, #1033, #1591
+
+We'll eventually replace this display with something more relevant when
+the new archiver code is ready.
+
+https://github.com/restic/restic/pull/1595

changelog/0.8.3_2018-02-26/issue-1497

@@ -0,0 +1,8 @@
+Enhancement: Add --read-data-subset flag to check command
+
+This change introduces ability to check integrity of a subset of repository 
+data packs. This can be used to spread integrity check of larger repositories
+over a period of time.
+
+https://github.com/restic/restic/issues/1497
+https://github.com/restic/restic/pull/1556

changelog/0.8.3_2018-02-26/issue-1633

@@ -0,0 +1,7 @@
+Bugfix: Fixed unexpected 'pack file cannot be listed' error
+
+Due to a regression introduced in 0.8.2, the `rebuild-index` and `prune`
+commands failed to read pack files with size of 587, 588, 589 or 590 bytes.
+
+https://github.com/restic/restic/issues/1633
+https://github.com/restic/restic/pull/1635

changelog/0.8.3_2018-02-26/issue-1641

@@ -0,0 +1,10 @@
+Bugfix: Ignore files with invalid names in the repo
+
+The release 0.8.2 introduced a bug: when restic encounters files in the repo
+which do not have a valid name, it tries to load a file with a name of lots of
+zeroes instead of ignoring it. This is now resolved, invalid file names are
+just ignored.
+
+https://github.com/restic/restic/issues/1641
+https://github.com/restic/restic/pull/1643
+https://forum.restic.net/t/help-fixing-repo-no-such-file/485/3

changelog/0.8.3_2018-02-26/pull-1560

@@ -0,0 +1,5 @@
+Enhancement: Retry all repository file download errors
+
+Restic will now retry failed downloads, similar to other operations.
+
+https://github.com/restic/restic/pull/1560

changelog/0.8.3_2018-02-26/pull-1623

@@ -0,0 +1,12 @@
+Enhancement: Don't check for presence of files in the backend before writing
+
+Before, all backend implementations were required to return an error if the
+file that is to be written already exists in the backend. For most backends,
+that means making a request (e.g. via HTTP) and returning an error when the
+file already exists.
+
+This is not accurate, the file could have been created between the HTTP request
+testing for it, and when writing starts, so we've relaxed this requeriment,
+which saves one additional HTTP request per newly added file.
+
+https://github.com/restic/restic/pull/1623

changelog/0.8.3_2018-02-26/pull-1634

@@ -0,0 +1,7 @@
+Enhancement: Upgrade B2 client library, reduce HTTP requests
+
+We've upgraded the B2 client library restic uses to access BackBlaze B2. This
+reduces the number of HTTP requests needed to upload a new file from two to
+one, which should improve throughput to B2.
+
+https://github.com/restic/restic/pull/1634