Update Gopkg.lock

repository/index: Avoid creating an error when checking if an id is in the index.

.github/ISSUE_TEMPLATE.md

@@ -53,3 +53,11 @@ this is possible), the easier it is for the project developers to fix it!
 
 
 ## Do you have an idea how to solve the issue?
+
+## Did restic help you or made you happy in any way?
+
+<!--
+Answering this question is not required, but if you have anything positive to share, please do so here!
+Sometimes we get tired of reading bug reports all day and a little positive end note does wonders.
+Idea by Joey Hess, https://joeyh.name/blog/entry/two_holiday_stories/
+-->

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,31 @@
+<!--
+Thank you very much for contributing code or documentation to restic! Please
+fill out the following questions to make it easier for us to review your
+changes.
+
+You do not need to check all the boxes below all at once, feel free to take
+your time and add more commits. If you're done and ready for review, please
+check the last box.
+-->
+
+### What is the purpose of this change? What does it change?
+
+<!--
+Describe the changes here, as detailed as needed.
+-->
+
+### Was the change discussed in an issue or in the forum before?
+
+<!--
+Link issues and relevant forum posts here.
+-->
+
+### Checklist
+
+- [ ] I have read the [Contribution Guidelines](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#providing-patches)
+- [ ] I have added tests for all changes in this PR
+- [ ] I have added documentation for the changes (in the manual)
+- [ ] There's a new file in a subdir of `changelog/x.y.z` that describe the changes for our users (template [here](https://github.com/restic/restic/blob/master/changelog/changelog-entry.tmpl))
+- [ ] I have run `gofmt` on the code in all commits
+- [ ] All commit messages are formatted in the same style as [the other commits in the repo](https://github.com/restic/restic/blob/master/CONTRIBUTING.md#git-commits)
+- [ ] I'm done, this Pull Request is ready for review

CHANGELOG.md

@@ -1,416 +1,700 @@
-This file describes changes relevant to all users that are made in each
+Changelog for restic 0.8.1 (2017-12-27)
-released version of restic from the perspective of the user.
+=======================================
 
-Important Changes in 0.8.0
+The following sections list the changes in restic 0.8.1 relevant to
-==========================
+restic users. The changes are ordered by importance.
 
- * A vulnerability was found in the restic restorer, which allowed attackers in
+Summary
-   special circumstances to restore files to a location outside of the target
+-------
-   directory. Due to the circumstances we estimate this to be a low-risk
-   vulnerability, but urge all users to upgrade to the latest version of restic.
 
-   Exploiting the vulnerability requires a Linux/Unix system which saves
+ * Fix #1457: Improve s3 backend with DigitalOcean Spaces
-   backups via restic and a Windows systems which restores files from the repo.
+ * Fix #1454: Correct cache dir location for Windows and Darwin
-   In addition, the attackers need to be able to create create files with
+ * Fix #1459: Disable handling SIGPIPE
-   arbitrary names which are then saved to the restic repo. For example, by
+ * Chg #1452: Do not save atime by default
-   creating a file named "..\test.txt" (which is a perfectly legal filename on
+ * Enh #1436: Add code to detect old cache directories
-   Linux) and restoring a snapshot containing this file on Windows, it would be
+ * Enh #1439: Improve cancellation logic
-   written to the parent of the target directory.
+ * Enh #11: Add the `diff` command
+
+Details
+-------
+
+ * Bugfix #1457: Improve s3 backend with DigitalOcean Spaces
+
+   https://github.com/restic/restic/issues/1457
+   https://github.com/restic/restic/pull/1459
+
+ * Bugfix #1454: Correct cache dir location for Windows and Darwin
+
+   The cache directory on Windows and Darwin was not correct, instead the directory `.cache` was
+   used.
+
+   https://github.com/restic/restic/pull/1454
+
+ * Bugfix #1459: Disable handling SIGPIPE
+
+   We've disabled handling SIGPIPE again. Turns out, writing to broken TCP connections also
+   raised SIGPIPE, so restic exits on the first write to a broken connection. Instead, restic
+   should retry the request.
+
+   https://github.com/restic/restic/issues/1457
+   https://github.com/restic/restic/issues/1466
+   https://github.com/restic/restic/pull/1459
+
+ * Change #1452: Do not save atime by default
+
+   By default, the access time for files and dirs is not saved any more. It is not possible to
+   reliably disable updating the access time during a backup, so for the next backup the access
+   time is different again. This means a lot of metadata is saved. If you want to save the access time
+   anyway, pass `--with-atime` to the `backup` command.
+
+   https://github.com/restic/restic/pull/1452
+
+ * Enhancement #1436: Add code to detect old cache directories
+
+   We've added code to detect old cache directories of repositories that haven't been used in a
+   long time, restic now prints a note when it detects that such dirs exist. Also, the option
+   `--cleanup-cache` was added to automatically remove such directories. That's not a problem
+   because the cache will be rebuild once a repo is accessed again.
+
+   https://github.com/restic/restic/pull/1436
+
+ * Enhancement #1439: Improve cancellation logic
+
+   The cancellation logic was improved, restic can now shut down cleanly when requested to do so
+   (e.g. via ctrl+c).
+
+   https://github.com/restic/restic/pull/1439
+
+ * Enhancement #11: Add the `diff` command
+
+   The command `diff` was added, it allows comparing two snapshots and listing all differences.
+
+   https://github.com/restic/restic/issues/11
+   https://github.com/restic/restic/issues/1460
+   https://github.com/restic/restic/pull/1462
+
+
+Changelog for restic 0.8.0 (2017-11-26)
+=======================================
+
+The following sections list the changes in restic 0.8.0 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Sec #1445: Prevent writing outside the target directory during restore
+ * Fix #1256: Re-enable workaround for S3 backend
+ * Fix #1291: Reuse backend TCP connections to BackBlaze B2
+ * Fix #1317: Run prune when `forget --prune` is called with just snapshot IDs
+ * Fix #1437: Remove implicit path `/restic` for the s3 backend
+ * Enh #1102: Add subdirectory `ids` to fuse mount
+ * Enh #1114: Add `--cacert` to specify TLS certificates to check against
+ * Enh #1216: Add upload/download limiting
+ * Enh #1271: Cache results for excludes for `backup`
+ * Enh #1274: Add `generate` command, replaces `manpage` and `autocomplete`
+ * Enh #1367: Allow comments in files read from via `--file-from`
+ * Enh #448: Sftp backend prompts for password
+ * Enh #510: Add `dump` command
+ * Enh #1040: Add local metadata cache
+ * Enh #1249: Add `latest` symlink in fuse mount
+ * Enh #1269: Add `--compact` to `forget` command
+ * Enh #1281: Google Cloud Storage backend needs less permissions
+ * Enh #1319: Make `check` print `no errors found` explicitly
+ * Enh #1353: Retry failed backend requests
+
+Details
+-------
+
+ * Security #1445: Prevent writing outside the target directory during restore
+
+   A vulnerability was found in the restic restorer, which allowed attackers in special
+   circumstances to restore files to a location outside of the target directory. Due to the
+   circumstances we estimate this to be a low-risk vulnerability, but urge all users to upgrade to
+   the latest version of restic.
+
+   Exploiting the vulnerability requires a Linux/Unix system which saves backups via restic and
+   a Windows systems which restores files from the repo. In addition, the attackers need to be able
+   to create create files with arbitrary names which are then saved to the restic repo. For
+   example, by creating a file named "..\test.txt" (which is a perfectly legal filename on Linux)
+   and restoring a snapshot containing this file on Windows, it would be written to the parent of
+   the target directory.
 
    We'd like to thank Tyler Spivey for reporting this responsibly!
 
    https://github.com/restic/restic/pull/1445
 
- * The s3 backend used the subdir `restic` within a bucket if no explicit path
+ * Bugfix #1256: Re-enable workaround for S3 backend
-   after the bucket name was specified. Since this version, restic does not use
-   this default path any more. If you created a repo on s3 in a bucket without
-   specifying a path within the bucket, you need to add `/restic` at the end of
-   the repository specification to access your repo: `s3:s3.amazonaws.com/bucket/restic`
-   https://github.com/restic/restic/issues/1292
-   https://github.com/restic/restic/pull/1437
 
- * We've added a local cache for metadata so that restic doesn't need to load
+   We've re-enabled a workaround for `minio-go` (the library we're using to access s3 backends),
-   all metadata (snapshots, indexes, ...) from the repo each time it starts. By
+   this reduces memory usage.
-   default the cache is active, but there's a new global option `--no-cache`
-   that can be used to disable the cache. By deafult, the cache a standard
-   cache folder for the OS, which can be overridden with `--cache-dir`.  The
-   cache will automatically populate, indexes and snapshots are saved as they
-   are loaded.
-   https://github.com/restic/restic/pull/1040
-   https://github.com/restic/restic/issues/29
-   https://github.com/restic/restic/issues/738
-   https://github.com/restic/restic/issues/282
-   https://github.com/restic/restic/pull/1287
 
- * A related change was to by default create pack files in the repo that
-   contain either data or metadata, not both mixed together. This allows easy
-   caching of only the metadata files. The next run of `restic prune` will
-   untangle mixed files automatically.
-   https://github.com/restic/restic/pull/1265
-
- * The Google Cloud Storage backend no longer requires the service account to
-   have the `storage.buckets.get` permission ("Storage Admin" role) in `restic
-   init` if the bucket already exists.
-   https://github.com/restic/restic/pull/1281
-
- * Added support for rate limiting through `--limit-upload` and
-   `--limit-download` flags.
-   https://github.com/restic/restic/issues/1216
-   https://github.com/restic/restic/pull/1336
-   https://github.com/restic/restic/pull/1358
-
- * Failed backend requests are now automatically retried.
-   https://github.com/restic/restic/pull/1353
-
- * We've added the `dump` command which prints a file from a snapshot to
-   stdout. This can e.g. be used to restore files read with `backup --stdin`.
-   https://github.com/restic/restic/issues/510
-   https://github.com/restic/restic/pull/1346
-
-Small changes
--------------
-
- * The directory structure in the fuse mount now exposes a symlink `latest`
-   which points to the latest snapshot in that particular directory.
-   https://github.com/restic/restic/pull/1249
-
- * The option `--compact` was added to the `forget` command to provide the same
-   compact view as the `snapshots` command.
-   https://github.com/restic/restic/pull/1269
-
- * We've re-enabled a workaround for `minio-go` (the library we're using to
-   access s3 backends), this reduces memory usage.
    https://github.com/restic/restic/issues/1256
    https://github.com/restic/restic/pull/1267
 
- * The sftp backend now prompts for the password if a password is necessary for
+ * Bugfix #1291: Reuse backend TCP connections to BackBlaze B2
-   login.
-   https://github.com/restic/restic/issues/448
-   https://github.com/restic/restic/pull/1270
 
- * The `generate` command has been added, which replaces the now removed
+   A bug was discovered in the library we're using to access Backblaze, it now reuses already
-   commands `manpage` and `autocomplete`. This release of restic contains the
+   established TCP connections which should be a lot faster and not cause network failures any
-   most recent manpages in `doc/man` and the auto-completion files for bash and
+   more.
-   zsh in `doc/bash-completion.sh` and `doc/zsh-completion.zsh`
-   https://github.com/restic/restic/issues/1274
-   https://github.com/restic/restic/pull/1282
 
- * A bug was discovered in the library we're using to access Backblaze, it now
-   reuses already established TCP connections which should be a lot faster and
-   not cause network failures any more.
    https://github.com/restic/restic/issues/1291
    https://github.com/restic/restic/pull/1301
 
- * Another bug in the `forget` command caused `prune` not to be run when
+ * Bugfix #1317: Run prune when `forget --prune` is called with just snapshot IDs
-   `--prune` was specified without a policy, e.g. when only snapshot IDs that
+
-   should be forgotten are listed manually. This is corrected now.
+   A bug in the `forget` command caused `prune` not to be run when `--prune` was specified without a
+   policy, e.g. when only snapshot IDs that should be forgotten are listed manually.
+
    https://github.com/restic/restic/pull/1317
 
- * The `check` command now explicetly prints `No errors were found` when no
+ * Bugfix #1437: Remove implicit path `/restic` for the s3 backend
-   errors could be found.
+
-   https://github.com/restic/restic/pull/1319
+   The s3 backend used the subdir `restic` within a bucket if no explicit path after the bucket name
-   https://github.com/restic/restic/issues/1303
+   was specified. Since this version, restic does not use this default path any more. If you
+   created a repo on s3 in a bucket without specifying a path within the bucket, you need to add
+   `/restic` at the end of the repository specification to access your repo:
+   `s3:s3.amazonaws.com/bucket/restic`
+
+   https://github.com/restic/restic/issues/1292
+   https://github.com/restic/restic/pull/1437
+
+ * Enhancement #1102: Add subdirectory `ids` to fuse mount
+
+   The fuse mount now has an `ids` subdirectory which contains the snapshots below their (short)
+   IDs.
 
- * The fuse mount now has an `ids` subdirectory which contains the snapshots
-   below their (short) IDs.
    https://github.com/restic/restic/issues/1102
    https://github.com/restic/restic/pull/1299
    https://github.com/restic/restic/pull/1320
 
- * The `backup` command was improved, it now caches the result of excludes for
+ * Enhancement #1114: Add `--cacert` to specify TLS certificates to check against
-   a directory.
+
-   https://github.com/restic/restic/issues/1271
+   We've added the `--cacert` option which can be used to pass one (or more) CA certificates to
-   https://github.com/restic/restic/pull/1326
+   restic. These are used in addition to the system CA certificates to verify HTTPS certificates
+   (e.g. for the REST backend).
 
- * We've added the `--cacert` option which can be used to pass one (or more) CA
-   certificates to restic. These are used in addition to the system CA
-   certificates to verify HTTPS certificates (e.g. for the REST backend).
    https://github.com/restic/restic/issues/1114
    https://github.com/restic/restic/pull/1276
 
- * When the list of files/dirs to be saved is read from a file with
+ * Enhancement #1216: Add upload/download limiting
-   `--files-from`, comment lines (starting with `#`) are now ignored.
+
+   We've added support for rate limiting through `--limit-upload` and `--limit-download`
+   flags.
+
+   https://github.com/restic/restic/issues/1216
+   https://github.com/restic/restic/pull/1336
+   https://github.com/restic/restic/pull/1358
+
+ * Enhancement #1271: Cache results for excludes for `backup`
+
+   The `backup` command now caches the result of excludes for a directory.
+
+   https://github.com/restic/restic/issues/1271
+   https://github.com/restic/restic/pull/1326
+
+ * Enhancement #1274: Add `generate` command, replaces `manpage` and `autocomplete`
+
+   The `generate` command has been added, which replaces the now removed commands `manpage` and
+   `autocomplete`. This release of restic contains the most recent manpages in `doc/man` and the
+   auto-completion files for bash and zsh in `doc/bash-completion.sh` and
+   `doc/zsh-completion.zsh`
+
+   https://github.com/restic/restic/issues/1274
+   https://github.com/restic/restic/pull/1282
+
+ * Enhancement #1367: Allow comments in files read from via `--file-from`
+
+   When the list of files/dirs to be saved is read from a file with `--files-from`, comment lines
+   (starting with `#`) are now ignored.
+
    https://github.com/restic/restic/issues/1367
    https://github.com/restic/restic/pull/1368
 
-Important Changes in 0.7.3
+ * Enhancement #448: Sftp backend prompts for password
-==========================
+
+   The sftp backend now prompts for the password if a password is necessary for login.
+
+   https://github.com/restic/restic/issues/448
+   https://github.com/restic/restic/pull/1270
+
+ * Enhancement #510: Add `dump` command
+
+   We've added the `dump` command which prints a file from a snapshot to stdout. This can e.g. be
+   used to restore files read with `backup --stdin`.
+
+   https://github.com/restic/restic/issues/510
+   https://github.com/restic/restic/pull/1346
+
+ * Enhancement #1040: Add local metadata cache
+
+   We've added a local cache for metadata so that restic doesn't need to load all metadata
+   (snapshots, indexes, ...) from the repo each time it starts. By default the cache is active, but
+   there's a new global option `--no-cache` that can be used to disable the cache. By deafult, the
+   cache a standard cache folder for the OS, which can be overridden with `--cache-dir`. The cache
+   will automatically populate, indexes and snapshots are saved as they are loaded. Cache
+   directories for repos that haven't been used recently can automatically be removed by restic
+   with the `--cleanup-cache` option.
+
+   A related change was to by default create pack files in the repo that contain either data or
+   metadata, not both mixed together. This allows easy caching of only the metadata files. The
+   next run of `restic prune` will untangle mixed files automatically.
+
+   https://github.com/restic/restic/issues/29
+   https://github.com/restic/restic/issues/738
+   https://github.com/restic/restic/issues/282
+   https://github.com/restic/restic/pull/1040
+   https://github.com/restic/restic/pull/1287
+   https://github.com/restic/restic/pull/1436
+   https://github.com/restic/restic/pull/1265
+
+ * Enhancement #1249: Add `latest` symlink in fuse mount
+
+   The directory structure in the fuse mount now exposes a symlink `latest` which points to the
+   latest snapshot in that particular directory.
+
+   https://github.com/restic/restic/pull/1249
+
+ * Enhancement #1269: Add `--compact` to `forget` command
+
+   The option `--compact` was added to the `forget` command to provide the same compact view as the
+   `snapshots` command.
+
+   https://github.com/restic/restic/pull/1269
+
+ * Enhancement #1281: Google Cloud Storage backend needs less permissions
+
+   The Google Cloud Storage backend no longer requires the service account to have the
+   `storage.buckets.get` permission ("Storage Admin" role) in `restic init` if the bucket
+   already exists.
+
+   https://github.com/restic/restic/pull/1281
+
+ * Enhancement #1319: Make `check` print `no errors found` explicitly
+
+   The `check` command now explicetly prints `No errors were found` when no errors could be found.
+
+   https://github.com/restic/restic/issues/1303
+   https://github.com/restic/restic/pull/1319
+
+ * Enhancement #1353: Retry failed backend requests
+
+   https://github.com/restic/restic/pull/1353
+
+
+Changelog for restic 0.7.3 (2017-09-20)
+=======================================
+
+The following sections list the changes in restic 0.7.3 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1246: List all files stored in Google Cloud Storage
+
+Details
+-------
+
+ * Bugfix #1246: List all files stored in Google Cloud Storage
+
+   For large backups stored in Google Cloud Storage, the `prune` command fails because listing
+   only returns the first 1000 files. This has been corrected, no data is lost in the process. In
+   addition, a plausibility check was added to `prune`.
 
- * For large backups stored in Google Cloud Storage, the `prune` command fails
-   because listing only returns the first 1000 files. This has been corrected,
-   no data is lost in the process. In addition, a plausibility check was added
-   to `prune`.
    https://github.com/restic/restic/issues/1246
    https://github.com/restic/restic/pull/1247
 
 
-Important Changes in 0.7.2
+Changelog for restic 0.7.2 (2017-09-13)
-==========================
+=======================================
 
- * We've added an official docker image and a Dockerfile to build this image in
+The following sections list the changes in restic 0.7.2 relevant to
-   `docker/`.
+restic users. The changes are ordered by importance.
-   https://github.com/restic/restic/pull/1061
 
- * The git repository layout was changed to resemble the layout typically used
+Summary
-   in Go projects, we're not using `gb` for building restic any more and
+-------
-   vendoring the dependencies is now taken care of by `dep`.
-   https://github.com/restic/restic/pull/1126
 
- * We now support saving backups on Google Cloud Storage.
+ * Fix #1167: Do not create a local repo unless `init` is used
-   https://github.com/restic/restic/pull/1134
+ * Fix #1164: Make the `key remove` command behave as documented
-   https://github.com/restic/restic/pull/1052
+ * Fix #1191: Make sure to write profiling files on interrupt
-   https://github.com/restic/restic/issues/211
+ * Enh #1132: Make `key` command always prompt for a password
+ * Enh #1179: Resolve name conflicts, append a counter
+ * Enh #1218: Add `--compact` to `snapshots` command
+ * Enh #317: Add `--exclude-caches` and `--exclude-if-present`
+ * Enh #697: Automatically generate man pages for all restic commands
+ * Enh #1044: Improve `restore`, do not traverse/load excluded directories
+ * Enh #1061: Add Dockerfile and official Docker image
+ * Enh #1126: Use the standard Go git repository layout, use `dep` for vendoring
+ * Enh #1134: Add support for storing backups on Google Cloud Storage
+ * Enh #1144: Properly report errors when reading files with exclude patterns.
+ * Enh #1149: Add support for storing backups on Microsoft Azure Blob Storage
+ * Enh #1196: Add `--group-by` to `forget` command for flexible grouping
+ * Enh #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
+ * Enh #1205: Allow specifying time/date for a backup with `--time`
 
- * We've added support for Microsoft Azure Blob Storage as a restic backend.
+Details
-   https://github.com/restic/restic/pull/1149
+-------
-   https://github.com/restic/restic/pull/1059
-   https://github.com/restic/restic/issues/609
 
- * In the course of supporting Microsoft Azure Blobe Storage Go 1.8 is now a
+ * Bugfix #1167: Do not create a local repo unless `init` is used
-   requirement to build restic.
 
- * The `restore` command has been improved: When dirs are excluded (or not
+   When a restic command other than `init` is used with a local repository and the repository
-   included) in a restore, they are not loaded from the repo any more.
+   directory does not exist, restic creates the directory structure. That's an error, only the
-   https://github.com/restic/restic/pull/1044
+   `init` command should create the dir.
 
- * Name collisions are now resolved by appending a counter.
-   https://github.com/restic/restic/issues/1179
-   https://github.com/restic/restic/pull/1209
-
-
-Small changes
--------------
-
- * The `key` command now prompts for a password even if the original password
-   to access a repo has been specified via the `RESTIC_PASSWORD` environment
-   variable or a password file.
-   https://github.com/restic/restic/issues/1132
-   https://github.com/restic/restic/pull/1133
-
- * Properly report errors when reading files with exclude patterns.
-   https://github.com/restic/restic/pull/1144
-
- * We now automatically generate man pages for all restic commands, see the
-   subdir `doc/man`.
-   https://github.com/restic/restic/issues/697
-   https://github.com/restic/restic/pull/1147
-
- * The `key remove` command was corrected and now works as documented.
-   https://github.com/restic/restic/pull/1164
-
- * When a restic command other than `init` is used with a local repository and
-   the repository directory does not exist, restic creates the directory
-   structure. That's an error, only the `init` command should create the dir.
    https://github.com/restic/restic/issues/1167
    https://github.com/restic/restic/pull/1182
 
- * Restic now prints stats on all BSD systems (not only on darwin) when SIGINFO
+ * Bugfix #1164: Make the `key remove` command behave as documented
-   is received (usually when ctrl+t is pressed).
+
-   https://github.com/restic/restic/pull/1203
+   https://github.com/restic/restic/pull/1164
-   https://github.com/restic/restic/pull/1082#issuecomment-326279920
+
+ * Bugfix #1191: Make sure to write profiling files on interrupt
+
+   Since a few releases restic had the ability to write profiling files for memory and CPU usage
+   when `debug` is enabled. It was discovered that when restic is interrupted (ctrl+c is
+   pressed), the proper shutdown hook is not run. This is now corrected.
 
- * Since a few releases restic had the ability to write profiling files for
-   memory and CPU usage when `debug` is enabled. It was discovered that when
-   restic is interrupted (ctrl+c is pressed), the proper shutdown hook is not
-   run. This is now corrected.
    https://github.com/restic/restic/pull/1191
 
- * A new option `--exclude-caches` was added that allows excluding cache
+ * Enhancement #1132: Make `key` command always prompt for a password
-   directories (that are tagged as such). This is a special case of a more
+
-   generic option `--exclude-if-present` which excludes a directory if a file
+   The `key` command now prompts for a password even if the original password to access a repo has
-   with a specific name (and contents) is present.
+   been specified via the `RESTIC_PASSWORD` environment variable or a password file.
+
+   https://github.com/restic/restic/issues/1132
+   https://github.com/restic/restic/pull/1133
+
+ * Enhancement #1179: Resolve name conflicts, append a counter
+
+   https://github.com/restic/restic/issues/1179
+   https://github.com/restic/restic/pull/1209
+
+ * Enhancement #1218: Add `--compact` to `snapshots` command
+
+   The option `--compact` was added to the `snapshots` command to get a better overview of the
+   snapshots in a repo. It limits each snapshot to a single line.
+
+   https://github.com/restic/restic/issues/1218
+   https://github.com/restic/restic/pull/1223
+
+ * Enhancement #317: Add `--exclude-caches` and `--exclude-if-present`
+
+   A new option `--exclude-caches` was added that allows excluding cache directories (that are
+   tagged as such). This is a special case of a more generic option `--exclude-if-present` which
+   excludes a directory if a file with a specific name (and contents) is present.
+
    https://github.com/restic/restic/issues/317
    https://github.com/restic/restic/pull/1170
    https://github.com/restic/restic/pull/1224
 
- * The `forget` command now has an option `--group-by` that allows flexible
+ * Enhancement #697: Automatically generate man pages for all restic commands
-   grouping policies.
+
+   https://github.com/restic/restic/issues/697
+   https://github.com/restic/restic/pull/1147
+
+ * Enhancement #1044: Improve `restore`, do not traverse/load excluded directories
+
+   https://github.com/restic/restic/pull/1044
+
+ * Enhancement #1061: Add Dockerfile and official Docker image
+
+   https://github.com/restic/restic/pull/1061
+
+ * Enhancement #1126: Use the standard Go git repository layout, use `dep` for vendoring
+
+   The git repository layout was changed to resemble the layout typically used in Go projects,
+   we're not using `gb` for building restic any more and vendoring the dependencies is now taken
+   care of by `dep`.
+
+   https://github.com/restic/restic/pull/1126
+
+ * Enhancement #1134: Add support for storing backups on Google Cloud Storage
+
+   https://github.com/restic/restic/issues/211
+   https://github.com/restic/restic/pull/1134
+   https://github.com/restic/restic/pull/1052
+
+ * Enhancement #1144: Properly report errors when reading files with exclude patterns.
+
+   https://github.com/restic/restic/pull/1144
+
+ * Enhancement #1149: Add support for storing backups on Microsoft Azure Blob Storage
+
+   The library we're using to access the service requires Go 1.8, so restic now needs at least Go
+   1.8.
+
+   https://github.com/restic/restic/issues/609
+   https://github.com/restic/restic/pull/1149
+   https://github.com/restic/restic/pull/1059
+
+ * Enhancement #1196: Add `--group-by` to `forget` command for flexible grouping
+
    https://github.com/restic/restic/pull/1196
 
- * The date and time restic records for a new backup can now be specified
+ * Enhancement #1203: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
-   externally by passing `--time` to the `backup` command.
+
+   https://github.com/restic/restic/pull/1203
+   https://github.com/restic/restic/pull/1082
+
+ * Enhancement #1205: Allow specifying time/date for a backup with `--time`
+
    https://github.com/restic/restic/pull/1205
 
- * The option `--compact` was added to the `snapshots` command to get a better
-   overview of the snapshots in a repo. It limits each snapshot to a single
-   line.
-   https://github.com/restic/restic/issues/1218
-   https://github.com/restic/restic/pull/1223
 
+Changelog for restic 0.7.1 (2017-07-22)
+=======================================
 
-Important Changes in 0.7.1
+The following sections list the changes in restic 0.7.1 relevant to
-==========================
+restic users. The changes are ordered by importance.
 
- * The `migrate` command for chaning the `s3legacy` layout to the `default`
+Summary
-   layout for s3 backends has been improved: It can now be restarted with
+-------
-   `restic migrate --force s3_layout` and automatically retries operations on
-   error.
-   https://github.com/restic/restic/issues/1073
-   https://github.com/restic/restic/pull/1075
 
-Small changes
+ * Fix #1115: Fix `prune`, only include existing files in indexes
--------------
+ * Enh #1055: Create subdirs below `data/` for local/sftp backends
+ * Enh #1067: Allow loading credentials for s3 from IAM
+ * Enh #1073: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
+ * Enh #1081: Clarify semantic for `--tasg` for the `forget` command
+ * Enh #1080: Ignore chmod() errors on filesystems which do not support it
+ * Enh #1082: Print stats on SIGINFO on Darwin and FreeBSD (ctrl+t)
+
+Details
+-------
+
+ * Bugfix #1115: Fix `prune`, only include existing files in indexes
+
+   A bug was found (and corrected) in the index rebuilding after prune, which led to indexes which
+   include blobs that were not present in the repo any more. There were already checks in place
+   which detected this situation and aborted with an error message. A new run of either `prune` or
+   `rebuild-index` corrected the index files. This is now fixed and a test has been added to detect
+   this.
+
+   https://github.com/restic/restic/pull/1115
+
+ * Enhancement #1055: Create subdirs below `data/` for local/sftp backends
+
+   The local and sftp backends now create the subdirs below `data/` on open/init. This way, restic
+   makes sure that they always exist. This is connected to an issue for the sftp server:
 
- * The local and sftp backends now create the subdirs below `data/` on
-   open/init. This way, restic makes sure that they always exist. This is
-   connected to an issue for the sftp server:
-   https://github.com/restic/rest-server/pull/11#issuecomment-309879710
    https://github.com/restic/restic/issues/1055
    https://github.com/restic/restic/pull/1077
    https://github.com/restic/restic/pull/1105
+   https://github.com/restic/rest-server/pull/11#issuecomment-309879710
+
+ * Enhancement #1067: Allow loading credentials for s3 from IAM
+
+   When no S3 credentials are specified in the environment variables, restic now tries to load
+   credentials from an IAM instance profile when the s3 backend is used.
 
- * When no S3 credentials are specified in the environment variables, restic
-   now tries to load credentials from an IAM instance profile when the s3
-   backend is used.
    https://github.com/restic/restic/issues/1067
    https://github.com/restic/restic/pull/1086
 
- * On Darwin and FreeBSD, restic now prints stats when SIGINFO is received
+ * Enhancement #1073: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
-   (usually when ctrl+t is pressed).
-   https://github.com/restic/restic/pull/1082
 
- * The dependencies have been updated.
+   The `migrate` command for chaning the `s3legacy` layout to the `default` layout for s3
-   https://github.com/restic/restic/pull/1108
+   backends has been improved: It can now be restarted with `restic migrate --force s3_layout`
-   https://github.com/restic/restic/pull/1124
+   and automatically retries operations on error.
 
- * A bug was found (and corrected) in the index rebuilding after prune, which
+   https://github.com/restic/restic/issues/1073
-   led to indexes which include blobs that were not present in the repo any
+   https://github.com/restic/restic/pull/1075
-   more. There were already checks in place which detected this situation and
-   aborted with an error message. A new run of either `prune` or
-   `rebuild-index` corrected the index files. This is now fixed and a test has
-   been added to detect this.
-   https://github.com/restic/restic/pull/1115
 
- * Errors for chmod() on Unix for filesystems which do not support it (e.g. smb
+ * Enhancement #1081: Clarify semantic for `--tasg` for the `forget` command
-   mounted via gvfs) are now ignored.
-   https://github.com/restic/restic/pull/1080
-   https://github.com/restic/restic/pull/1112
 
- * The semantic for the `--tags` option to `forget` and `snapshots` was
-   clarified:
    https://github.com/restic/restic/issues/1081
    https://github.com/restic/restic/pull/1090
 
-Important Changes in 0.7.0
+ * Enhancement #1080: Ignore chmod() errors on filesystems which do not support it
-==========================
 
- * New "swift" backend: A new backend for the OpenStack Swift cloud storage
+   https://github.com/restic/restic/pull/1080
-   protocol has been added, https://wiki.openstack.org/wiki/Swift
+   https://github.com/restic/restic/pull/1112
-   https://github.com/restic/restic/pull/975
-   https://github.com/restic/restic/pull/648
 
- * New "b2" backend: A new backend for Backblaze B2 cloud storage
+ * Enhancement #1082: Print stats on SIGINFO on Darwin and FreeBSD (ctrl+t)
-   service has been added, https://www.backblaze.com
-   https://github.com/restic/restic/issues/512
-   https://github.com/restic/restic/pull/978
 
- * Improved performance for the `find` command: Restic recognizes paths it has
+   https://github.com/restic/restic/pull/1082
-   already checked for the files in question, so the number of backend requests
-   is reduced a lot.
-   https://github.com/restic/restic/issues/989
-   https://github.com/restic/restic/pull/993
 
- * Improved performance for the fuse mount: Listing directories which contain
-   large files now is significantly faster.
-   https://github.com/restic/restic/pull/998
 
- * The default layout for the s3 backend is now `default` (instead of
+Changelog for restic 0.7.0 (2017-07-01)
-   `s3legacy`). Also, there's a new `migrate` command to convert an existing
+=======================================
-   repo, it can be run like this: `restic migrate s3_layout`
-   https://github.com/restic/restic/issues/965
-   https://github.com/restic/restic/pull/1004
 
- * The fuse mount now has two more directories: `tags` contains a subdir for
+The following sections list the changes in restic 0.7.0 relevant to
-   each tag, which in turn contains only the snapshots that have this tag. The
+restic users. The changes are ordered by importance.
-   subdir `hosts` contains a subdir for each host that has a snapshot, and the
-   subdir contains the snapshots for that host.
-   https://github.com/restic/restic/issues/636
-   https://github.com/restic/restic/pull/1050
 
-Small changes
+Summary
--------------
+-------
+
+ * Fix #1013: Switch back to using the high-level minio-go API for s3
+ * Fix #965: Switch to `default` repo layout for the s3 backend
+ * Enh #1021: Detect invalid backend name and print error
+ * Enh #1029: Remove invalid pack files when `prune` is run
+ * Enh #512: Add Backblaze B2 backend
+ * Enh #636: Add dirs `tags` and `hosts` to fuse mount
+ * Enh #989: Improve performance of the `find` command
+ * Enh #975: Add new backend for OpenStack Swift
+ * Enh #998: Improve performance of the fuse mount
+
+Details
+-------
+
+ * Bugfix #1013: Switch back to using the high-level minio-go API for s3
+
+   For the s3 backend we're back to using the high-level API the s3 client library for uploading
+   data, a few users reported dropped connections (which the library will automatically retry
+   now).
 
- * For the s3 backend we're back to using the high-level API the s3 client
-   library for uploading data, a few users reported dropped connections (which
-   the library will automatically retry now).
    https://github.com/restic/restic/issues/1013
    https://github.com/restic/restic/issues/1023
    https://github.com/restic/restic/pull/1025
 
- * The `prune` command has been improved and will now remove invalid pack
+ * Bugfix #965: Switch to `default` repo layout for the s3 backend
-   files, for example files that have not been uploaded completely because a
+
-   backup was interrupted.
+   The default layout for the s3 backend is now `default` (instead of `s3legacy`). Also, there's a
-   https://github.com/restic/restic/issues/1029
+   new `migrate` command to convert an existing repo, it can be run like this: `restic migrate
-   https://github.com/restic/restic/pull/1036
+   s3_layout`
+
+   https://github.com/restic/restic/issues/965
+   https://github.com/restic/restic/pull/1004
+
+ * Enhancement #1021: Detect invalid backend name and print error
+
+   Restic now tries to detect when an invalid/unknown backend is used and returns an error
+   message.
 
- * restic now tries to detect when an invalid/unknown backend is used and
-   returns an error message.
    https://github.com/restic/restic/issues/1021
    https://github.com/restic/restic/pull/1070
 
-Important Changes in 0.6.1
+ * Enhancement #1029: Remove invalid pack files when `prune` is run
-==========================
 
-This is mostly a bugfix release and only contains small changes:
+   The `prune` command has been improved and will now remove invalid pack files, for example files
+   that have not been uploaded completely because a backup was interrupted.
 
- * We've fixed a bug where `rebuild-index` would corrupt the index when used
+   https://github.com/restic/restic/issues/1029
-   with the s3 backend together with the `default` layout. This is not the
+   https://github.com/restic/restic/pull/1036
-   default setting.
+
+ * Enhancement #512: Add Backblaze B2 backend
+
+   https://github.com/restic/restic/issues/512
+   https://github.com/restic/restic/pull/978
+
+ * Enhancement #636: Add dirs `tags` and `hosts` to fuse mount
+
+   The fuse mount now has two more directories: `tags` contains a subdir for each tag, which in turn
+   contains only the snapshots that have this tag. The subdir `hosts` contains a subdir for each
+   host that has a snapshot, and the subdir contains the snapshots for that host.
+
+   https://github.com/restic/restic/issues/636
+   https://github.com/restic/restic/pull/1050
+
+ * Enhancement #989: Improve performance of the `find` command
+
+   Improved performance for the `find` command: Restic recognizes paths it has already checked
+   for the files in question, so the number of backend requests is reduced a lot.
+
+   https://github.com/restic/restic/issues/989
+   https://github.com/restic/restic/pull/993
+
+ * Enhancement #975: Add new backend for OpenStack Swift
+
+   https://github.com/restic/restic/pull/975
+   https://github.com/restic/restic/pull/648
+
+ * Enhancement #998: Improve performance of the fuse mount
+
+   Listing directories which contain large files now is significantly faster.
+
+   https://github.com/restic/restic/pull/998
+
+
+Changelog for restic 0.6.1 (2017-06-01)
+=======================================
+
+The following sections list the changes in restic 0.6.1 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Enh #985: Allow multiple parallel idle HTTP connections
+ * Enh #981: Remove temporary path from binary in `build.go`
+ * Enh #974: Remove regular status reports
+
+Details
+-------
+
+ * Enhancement #985: Allow multiple parallel idle HTTP connections
+
+   Backends based on HTTP now allow several idle connections in parallel. This is especially
+   important for the REST backend, which (when used with a local server) may create a lot
+   connections and exhaust available ports quickly.
 
- * Backends based on HTTP now allow several idle connections in parallel. This
-   is especially important for the REST backend, which (when used with a local
-   server) may create a lot connections and exhaust available ports quickly.
    https://github.com/restic/restic/issues/985
    https://github.com/restic/restic/pull/986
 
- * Regular status report: We've removed the status report that was printed
+ * Enhancement #981: Remove temporary path from binary in `build.go`
-   every 10 seconds when restic is run non-interactively. You can still force
+
-   reporting the current status by sending a `USR1` signal to the process.
+   The `build.go` now strips the temporary directory used for compilation from the binary. This
-   https://github.com/restic/restic/pull/974
+   is the first step in enabling reproducible builds.
 
- * The `build.go` now strips the temporary directory used for compilation from
-   the binary. This is the first step in enabling reproducible builds.
    https://github.com/restic/restic/pull/981
 
-Important Changes in 0.6.0
+ * Enhancement #974: Remove regular status reports
-==========================
 
-Consistent forget policy
+   Regular status report: We've removed the status report that was printed every 10 seconds when
-------------------------
+   restic is run non-interactively. You can still force reporting the current status by sending a
+   `USR1` signal to the process.
 
-The `forget` command was corrected to be more consistent in which snapshots are
+   https://github.com/restic/restic/pull/974
-to be forgotten. It is possible that the new code removes more snapshots than
+
-before, so please review what would be deleted by using the `--dry-run` option.
+
+Changelog for restic 0.6.0 (2017-05-29)
+=======================================
+
+The following sections list the changes in restic 0.6.0 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Enh #957: Make `forget` consistent
+ * Enh #966: Unify repository layout for all backends
+ * Enh #962: Improve memory and runtime for the s3 backend
+
+Details
+-------
+
+ * Enhancement #957: Make `forget` consistent
+
+   The `forget` command was corrected to be more consistent in which snapshots are to be
+   forgotten. It is possible that the new code removes more snapshots than before, so please
+   review what would be deleted by using the `--dry-run` option.
 
-https://github.com/restic/restic/pull/957
    https://github.com/restic/restic/issues/953
+   https://github.com/restic/restic/pull/957
 
-Unified repository layout
+ * Enhancement #966: Unify repository layout for all backends
--------------------------
 
-Up to now the s3 backend used a special repository layout. We've decided to
+   Up to now the s3 backend used a special repository layout. We've decided to unify the repository
-unify the repository layout and implemented the default layout also for the s3
+   layout and implemented the default layout also for the s3 backend. For creating a new
-backend. For creating a new repository on s3 with the default layout, use
+   repository on s3 with the default layout, use `restic -o s3.layout=default init`. For further
-`restic -o s3.layout=default init`. For further commands the option is not
+   commands the option is not necessary any more, restic will automatically detect the correct
-necessary any more, restic will automatically detect the correct layout to use.
+   layout to use. A future version will switch to the default layout for new repositories.
-A future version will switch to the default layout for new repositories.
 
-https://github.com/restic/restic/pull/966
    https://github.com/restic/restic/issues/965
+   https://github.com/restic/restic/pull/966
 
-Memory and time improvements for the s3 backend
+ * Enhancement #962: Improve memory and runtime for the s3 backend
------------------------------------------------
 
-We've updated the library used for accessing s3, switched to using a lower
+   We've updated the library used for accessing s3, switched to using a lower level API and added
-level API and added caching for some requests. This lead to a decrease in
+   caching for some requests. This lead to a decrease in memory usage and a great speedup. In
-memory usage and a great speedup. In addition, we added benchmark functions for
+   addition, we added benchmark functions for all backends, so we can track improvements over
-all backends, so we can track improvements over time. The Continuous
+   time. The Continuous Integration test service we're using (Travis) now runs the s3 backend
-Integration test service we're using (Travis) now runs the s3 backend tests not
+   tests not only against a Minio server, but also against the Amazon s3 live service, so we should
-only against a Minio server, but also against the Amazon s3 live service, so we
+   be notified of any regressions much sooner.
-should be notified of any regressions much sooner.
 
    https://github.com/restic/restic/pull/962
    https://github.com/restic/restic/pull/960
    https://github.com/restic/restic/pull/946
    https://github.com/restic/restic/pull/938
    https://github.com/restic/restic/pull/883
+
+

Gopkg.lock

@@ -10,20 +10,20 @@
 [[projects]]
   name = "cloud.google.com/go"
   packages = ["compute/metadata"]
-  revision = "eaddaf6dd7ee35fd3c2420c8d27478db176b0485"
+  revision = "2d3a6656c17a60b0815b7e06ab0be04eacb6e613"
-  version = "v0.15.0"
+  version = "v0.16.0"
 
 [[projects]]
   name = "github.com/Azure/azure-sdk-for-go"
   packages = ["storage"]
-  revision = "509eea43b93cec2f3f17acbe2578ef58703923f8"
+  revision = "7692b0cef22674113fcf71cc17ac3ccc1a7fef48"
-  version = "v11.1.1-beta"
+  version = "v11.2.2-beta"
 
 [[projects]]
   name = "github.com/Azure/go-autorest"
   packages = ["autorest","autorest/adal","autorest/azure","autorest/date"]
-  revision = "7aa5b8a6f18b5c15910c767ab005fc4585221177"
+  revision = "c67b24a8e30d876542a85022ebbdecf0e5a935e8"
-  version = "v9.1.1"
+  version = "v9.4.1"
 
 [[projects]]
   name = "github.com/cenkalti/backoff"
@@ -47,7 +47,7 @@
   branch = "master"
   name = "github.com/dustin/go-humanize"
   packages = ["."]
-  revision = "77ed807830b4df581417e7f89eb81d4872832b72"
+  revision = "bb3d318650d48840a39aa21a027c6630e198e626"
 
 [[projects]]
   name = "github.com/elithrar/simple-scrypt"
@@ -58,14 +58,14 @@
 [[projects]]
   name = "github.com/go-ini/ini"
   packages = ["."]
-  revision = "5b3e00af70a9484542169a976dcab8d03e601a17"
+  revision = "32e4c1e6bc4e7d0d8451aa6b75200d19e37a536a"
-  version = "v1.30.0"
+  version = "v1.32.0"
 
 [[projects]]
   branch = "master"
   name = "github.com/golang/protobuf"
   packages = ["proto"]
-  revision = "1643683e1b54a9e88ad26d98f81400c8c9d9f4f9"
+  revision = "1e59b77b52bf8e4b449a57e6f79f21226d571845"
 
 [[projects]]
   name = "github.com/inconshreveable/mousetrap"
@@ -77,7 +77,7 @@
   branch = "master"
   name = "github.com/juju/ratelimit"
   packages = ["."]
-  revision = "5b9ff866471762aa2ab2dced63c9fb6f53921342"
+  revision = "59fac5042749a5afb9af70e813da1dd5474f0167"
 
 [[projects]]
   branch = "master"
@@ -100,8 +100,8 @@
 [[projects]]
   name = "github.com/minio/minio-go"
   packages = [".","pkg/credentials","pkg/encrypt","pkg/policy","pkg/s3signer","pkg/s3utils","pkg/set"]
-  revision = "4e0f567303d4cc90ceb055a451959fb9fc391fb9"
+  revision = "57a8ae886b49af6eb0d2c27c2d007ed2f71e1da5"
-  version = "3.0.3"
+  version = "4.0.3"
 
 [[projects]]
   branch = "master"
@@ -158,10 +158,10 @@
   version = "v1.0.3"
 
 [[projects]]
-  branch = "master"
   name = "github.com/spf13/cobra"
   packages = [".","doc"]
   revision = "7b2c5ac9fc04fc5efafb60700713d4fa609b777b"
+  version = "v0.0.1"
 
 [[projects]]
   name = "github.com/spf13/pflag"
@@ -173,31 +173,31 @@
   branch = "master"
   name = "golang.org/x/crypto"
   packages = ["curve25519","ed25519","ed25519/internal/edwards25519","pbkdf2","poly1305","scrypt","ssh","ssh/terminal"]
-  revision = "edd5e9b0879d13ee6970a50153d85b8fec9f7686"
+  revision = "94eea52f7b742c7cbe0b03b22f0c4c8631ece122"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/net"
-  packages = ["context","context/ctxhttp"]
+  packages = ["context","context/ctxhttp","webdav","webdav/internal/xml"]
-  revision = "cd69bc3fc700721b709c3a59e16e24c67b58f6ff"
+  revision = "a8b9294777976932365dabb6640cf1468d95c70f"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/oauth2"
   packages = [".","google","internal","jws","jwt"]
-  revision = "bb50c06baba3d0c76f9d125c0719093e315b5b44"
+  revision = "f95fa95eaa936d9d87489b15d1d18b97c1ba9c28"
 
 [[projects]]
   branch = "master"
   name = "golang.org/x/sys"
   packages = ["unix","windows"]
-  revision = "8dbc5d05d6edcc104950cc299a1ce6641235bc86"
+  revision = "8b4580aae2a0dd0c231a45d3ccb8434ff533b840"
 
 [[projects]]
   branch = "master"
   name = "google.golang.org/api"
   packages = ["gensupport","googleapi","googleapi/internal/uritemplates","storage/v1"]
-  revision = "7afc123cf726cd2f253faa3e144d2ab65477b18f"
+  revision = "3a1d936b7575b82197a1fea0632218dd07b1e65c"
 
 [[projects]]
   name = "google.golang.org/appengine"
@@ -209,11 +209,11 @@
   branch = "v2"
   name = "gopkg.in/yaml.v2"
   packages = ["."]
-  revision = "eb3733d160e74a9c7e442f435eb3bea458e1d19f"
+  revision = "287cf08546ab5e7e37d55a84f7ed3fd1db036de5"
 
 [solve-meta]
   analyzer-name = "dep"
   analyzer-version = 1
-  inputs-digest = "f0a207197cb502238ac87ca8e07b2640c02ec380a50b036e09ef87e40e31ca2d"
+  inputs-digest = "b01eeeb2be041c7cd11f9ee50324ef456ac1e1cd0720408c6d72f88f92f09320"
   solver-name = "gps-cdcl"
   solver-version = 1

README.rst

@@ -51,7 +51,7 @@ Therefore, restic supports the following backends for storing backups natively:
 
 - `Local directory <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#local>`__
 - `sftp server (via SSH) <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#sftp>`__
-- `HTTP REST server <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server>`__ (`protocol <doc/rest_backend.rst>`__ `rest-server <https://github.com/restic/rest-server>`__)
+- `HTTP REST server <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#rest-server>`__ (`protocol <doc/100_references.rst#rest-backend>`__ `rest-server <https://github.com/restic/rest-server>`__)
 - `AWS S3 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#amazon-s3>`__ (either from Amazon or using the `Minio <https://minio.io>`__ server)
 - `OpenStack Swift <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#openstack-swift>`__
 - `BackBlaze B2 <https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2>`__
@@ -102,7 +102,7 @@ News
 ----
 
 You can follow the restic project on Twitter `@resticbackup <https://twitter.com/resticbackup>`__ or by subscribing to
-the `development blog <https://restic.github.io/blog/>`__.
+the `development blog <https://restic.net/blog/>`__.
 
 License
 -------

VERSION

@@ -1 +1 @@
-0.8.0
+0.8.1

build.go

@@ -11,6 +11,7 @@ import (
 	"path"
 	"path/filepath"
 	"runtime"
+	"strconv"
 	"strings"
 )
 
@@ -26,6 +27,7 @@ var config = struct {
 	Namespace  string
 	Main       string
 	Tests      []string
+	MinVersion GoVersion
 }{
 	Name:      "restic",                              // name of the program executable and directory
 	Namespace: "github.com/restic/restic",            // subdir of GOPATH, e.g. "github.com/foo/bar"
@@ -33,6 +35,7 @@ var config = struct {
 	Tests: []string{ // tests to run
 		"github.com/restic/restic/internal/...",
 		"github.com/restic/restic/cmd/..."},
+	MinVersion: GoVersion{Major: 1, Minor: 8, Patch: 0}, // minimum Go version supported
 }
 
 // specialDir returns true if the file begins with a special character ('.' or '_').
@@ -137,7 +140,6 @@ func copyFile(dst, src string) error {
 	if err != nil {
 		return err
 	}
-	defer fsrc.Close()
 
 	if err = os.MkdirAll(filepath.Dir(dst), 0755); err != nil {
 		fmt.Printf("MkdirAll(%v)\n", filepath.Dir(dst))
@@ -148,17 +150,28 @@ func copyFile(dst, src string) error {
 	if err != nil {
 		return err
 	}
-	defer fdst.Close()
 
-	_, err = io.Copy(fdst, fsrc)
+	if _, err = io.Copy(fdst, fsrc); err != nil {
+		return err
+	}
+
+	if err == nil {
+		err = fsrc.Close()
+	}
+
+	if err == nil {
+		err = fdst.Close()
+	}
+
 	if err == nil {
 		err = os.Chmod(dst, fi.Mode())
 	}
+
 	if err == nil {
 		err = os.Chtimes(dst, fi.ModTime(), fi.ModTime())
 	}
 
-	return err
+	return nil
 }
 
 // die prints the message with fmt.Fprintf() to stderr and exits with an error
@@ -300,10 +313,80 @@ func (cs Constants) LDFlags() string {
 	return strings.Join(l, " ")
 }
 
+// GoVersion is the version of Go used to compile the project.
+type GoVersion struct {
+	Major int
+	Minor int
+	Patch int
+}
+
+// ParseGoVersion parses the Go version s. If s cannot be parsed, the returned GoVersion is null.
+func ParseGoVersion(s string) (v GoVersion) {
+	if !strings.HasPrefix(s, "go") {
+		return
+	}
+
+	s = s[2:]
+	data := strings.Split(s, ".")
+	if len(data) != 3 {
+		return
+	}
+
+	major, err := strconv.Atoi(data[0])
+	if err != nil {
+		return
+	}
+
+	minor, err := strconv.Atoi(data[1])
+	if err != nil {
+		return
+	}
+
+	patch, err := strconv.Atoi(data[2])
+	if err != nil {
+		return
+	}
+
+	v = GoVersion{
+		Major: major,
+		Minor: minor,
+		Patch: patch,
+	}
+	return
+}
+
+// AtLeast returns true if v is at least as new as other. If v is empty, true is returned.
+func (v GoVersion) AtLeast(other GoVersion) bool {
+	var empty GoVersion
+
+	// the empty version satisfies all versions
+	if v == empty {
+		return true
+	}
+
+	if v.Major < other.Major {
+		return false
+	}
+
+	if v.Minor < other.Minor {
+		return false
+	}
+
+	if v.Patch < other.Patch {
+		return false
+	}
+
+	return true
+}
+
+func (v GoVersion) String() string {
+	return fmt.Sprintf("Go %d.%d.%d", v.Major, v.Minor, v.Patch)
+}
+
 func main() {
-	ver := runtime.Version()
+	ver := ParseGoVersion(runtime.Version())
-	if strings.HasPrefix(ver, "go1") && ver < "go1.8" {
+	if !ver.AtLeast(config.MinVersion) {
-		fmt.Fprintf(os.Stderr, "Go version %s detected, restic requires at least Go 1.8\n", ver)
+		fmt.Fprintf(os.Stderr, "%s detected, this program requires at least %s\n", ver, config.MinVersion)
 		os.Exit(1)
 	}
 

changelog/0.6.0/issue-953

@@ -0,0 +1,8 @@
+Enhancement: Make `forget` consistent
+
+The `forget` command was corrected to be more consistent in which snapshots are
+to be forgotten. It is possible that the new code removes more snapshots than
+before, so please review what would be deleted by using the `--dry-run` option.
+
+https://github.com/restic/restic/pull/957
+https://github.com/restic/restic/issues/953

changelog/0.6.0/issue-965

@@ -0,0 +1,11 @@
+Enhancement: Unify repository layout for all backends
+
+Up to now the s3 backend used a special repository layout. We've decided to
+unify the repository layout and implemented the default layout also for the s3
+backend. For creating a new repository on s3 with the default layout, use
+`restic -o s3.layout=default init`. For further commands the option is not
+necessary any more, restic will automatically detect the correct layout to use.
+A future version will switch to the default layout for new repositories.
+
+https://github.com/restic/restic/pull/966
+https://github.com/restic/restic/issues/965

changelog/0.6.0/pull-962

@@ -0,0 +1,15 @@
+Enhancement: Improve memory and runtime for the s3 backend
+
+We've updated the library used for accessing s3, switched to using a lower
+level API and added caching for some requests. This lead to a decrease in
+memory usage and a great speedup. In addition, we added benchmark functions for
+all backends, so we can track improvements over time. The Continuous
+Integration test service we're using (Travis) now runs the s3 backend tests not
+only against a Minio server, but also against the Amazon s3 live service, so we
+should be notified of any regressions much sooner.
+
+https://github.com/restic/restic/pull/962
+https://github.com/restic/restic/pull/960
+https://github.com/restic/restic/pull/946
+https://github.com/restic/restic/pull/938
+https://github.com/restic/restic/pull/883

changelog/0.6.1/issue-985

@@ -0,0 +1,8 @@
+Enhancement: Allow multiple parallel idle HTTP connections
+
+Backends based on HTTP now allow several idle connections in parallel. This
+is especially important for the REST backend, which (when used with a local
+server) may create a lot connections and exhaust available ports quickly.
+
+https://github.com/restic/restic/issues/985
+https://github.com/restic/restic/pull/986

changelog/0.6.1/pull-891

@@ -0,0 +1,6 @@
+Enhancement: Remove temporary path from binary in `build.go`
+
+The `build.go` now strips the temporary directory used for compilation from
+the binary. This is the first step in enabling reproducible builds.
+
+https://github.com/restic/restic/pull/981

changelog/0.6.1/pull-974

@@ -0,0 +1,7 @@
+Enhancement: Remove regular status reports
+
+Regular status report: We've removed the status report that was printed
+every 10 seconds when restic is run non-interactively. You can still force
+reporting the current status by sending a `USR1` signal to the process.
+
+https://github.com/restic/restic/pull/974

changelog/0.7.0/issue-1013

@@ -0,0 +1,9 @@
+Bugfix: Switch back to using the high-level minio-go API for s3
+
+For the s3 backend we're back to using the high-level API the s3 client library
+for uploading data, a few users reported dropped connections (which the library
+will automatically retry now).
+
+https://github.com/restic/restic/issues/1013
+https://github.com/restic/restic/issues/1023
+https://github.com/restic/restic/pull/1025

changelog/0.7.0/issue-1021

@@ -0,0 +1,7 @@
+Enhancement: Detect invalid backend name and print error
+
+restic now tries to detect when an invalid/unknown backend is used and
+returns an error message.
+
+https://github.com/restic/restic/issues/1021
+https://github.com/restic/restic/pull/1070

changelog/0.7.0/issue-1029

@@ -0,0 +1,8 @@
+Enhancement: Remove invalid pack files when `prune` is run
+
+The `prune` command has been improved and will now remove invalid pack files,
+for example files that have not been uploaded completely because a backup was
+interrupted.
+
+https://github.com/restic/restic/issues/1029
+https://github.com/restic/restic/pull/1036

changelog/0.7.0/issue-512

@@ -0,0 +1,4 @@
+Enhancement: Add Backblaze B2 backend
+
+https://github.com/restic/restic/issues/512
+https://github.com/restic/restic/pull/978

changelog/0.7.0/issue-636

@@ -0,0 +1,9 @@
+Enhancement: Add dirs `tags` and `hosts` to fuse mount
+
+The fuse mount now has two more directories: `tags` contains a subdir for
+each tag, which in turn contains only the snapshots that have this tag. The
+subdir `hosts` contains a subdir for each host that has a snapshot, and the
+subdir contains the snapshots for that host.
+
+https://github.com/restic/restic/issues/636
+https://github.com/restic/restic/pull/1050

changelog/0.7.0/issue-965

@@ -0,0 +1,8 @@
+Bugfix: Switch to `default` repo layout for the s3 backend
+
+The default layout for the s3 backend is now `default` (instead of `s3legacy`).
+Also, there's a new `migrate` command to convert an existing repo, it can be
+run like this: `restic migrate s3_layout`
+
+https://github.com/restic/restic/issues/965
+https://github.com/restic/restic/pull/1004

changelog/0.7.0/issue-989

@@ -0,0 +1,8 @@
+Enhancement: Improve performance of the `find` command
+
+Improved performance for the `find` command: Restic recognizes paths it has
+already checked for the files in question, so the number of backend requests
+is reduced a lot.
+
+https://github.com/restic/restic/issues/989
+https://github.com/restic/restic/pull/993

changelog/0.7.0/pull-975

@@ -0,0 +1,4 @@
+Enhancement: Add new backend for OpenStack Swift
+
+https://github.com/restic/restic/pull/975
+https://github.com/restic/restic/pull/648

changelog/0.7.0/pull-998

@@ -0,0 +1,5 @@
+Enhancement: Improve performance of the fuse mount
+
+Listing directories which contain large files now is significantly faster.
+
+https://github.com/restic/restic/pull/998

changelog/0.7.1/issue-1055

@@ -0,0 +1,10 @@
+Enhancement: Create subdirs below `data/` for local/sftp backends
+
+The local and sftp backends now create the subdirs below `data/` on
+open/init. This way, restic makes sure that they always exist. This is
+connected to an issue for the sftp server:
+
+https://github.com/restic/rest-server/pull/11#issuecomment-309879710
+https://github.com/restic/restic/issues/1055
+https://github.com/restic/restic/pull/1077
+https://github.com/restic/restic/pull/1105

changelog/0.7.1/issue-1067

@@ -0,0 +1,8 @@
+Enhancement: Allow loading credentials for s3 from IAM
+
+When no S3 credentials are specified in the environment variables, restic
+now tries to load credentials from an IAM instance profile when the s3
+backend is used.
+
+https://github.com/restic/restic/issues/1067
+https://github.com/restic/restic/pull/1086

changelog/0.7.1/issue-1073

@@ -0,0 +1,8 @@
+Enhancement: Add `migrate` cmd to migrate from `s3legacy` to `default` layout
+
+The `migrate` command for chaning the `s3legacy` layout to the `default` layout
+for s3 backends has been improved: It can now be restarted with `restic migrate
+--force s3_layout` and automatically retries operations on error.
+
+https://github.com/restic/restic/issues/1073
+https://github.com/restic/restic/pull/1075

changelog/0.7.1/issue-1081

@@ -0,0 +1,4 @@
+Enhancement: Clarify semantic for `--tasg` for the `forget` command
+
+https://github.com/restic/restic/issues/1081
+https://github.com/restic/restic/pull/1090

changelog/0.7.1/plul-1080

@@ -0,0 +1,5 @@
+Enhancement: Ignore chmod() errors on filesystems which do not support it
+
+https://github.com/restic/restic/pull/1080
+https://github.com/restic/restic/pull/1112
+

changelog/0.7.1/pull-1082

@@ -0,0 +1,3 @@
+Enhancement: Print stats on SIGINFO on Darwin and FreeBSD (ctrl+t)
+
+https://github.com/restic/restic/pull/1082

changelog/0.7.1/pull-1115

@@ -0,0 +1,9 @@
+Bugfix: Fix `prune`, only include existing files in indexes
+
+A bug was found (and corrected) in the index rebuilding after prune, which led
+to indexes which include blobs that were not present in the repo any more.
+There were already checks in place which detected this situation and aborted
+with an error message. A new run of either `prune` or `rebuild-index` corrected
+the index files. This is now fixed and a test has been added to detect this.
+
+https://github.com/restic/restic/pull/1115

changelog/0.7.2/issue-1132

@@ -0,0 +1,8 @@
+Enhancement: Make `key` command always prompt for a password
+
+The `key` command now prompts for a password even if the original password
+to access a repo has been specified via the `RESTIC_PASSWORD` environment
+variable or a password file.
+
+https://github.com/restic/restic/issues/1132
+https://github.com/restic/restic/pull/1133

changelog/0.7.2/issue-1167

@@ -0,0 +1,8 @@
+Bugfix: Do not create a local repo unless `init` is used
+
+When a restic command other than `init` is used with a local repository and the
+repository directory does not exist, restic creates the directory structure.
+That's an error, only the `init` command should create the dir.
+
+https://github.com/restic/restic/issues/1167
+https://github.com/restic/restic/pull/1182

changelog/0.7.2/issue-1179

@@ -0,0 +1,4 @@
+Enhancement: Resolve name conflicts, append a counter
+
+https://github.com/restic/restic/issues/1179
+https://github.com/restic/restic/pull/1209

changelog/0.7.2/issue-1208

@@ -0,0 +1,7 @@
+Enhancement: Add `--compact` to `snapshots` command
+
+The option `--compact` was added to the `snapshots` command to get a better
+overview of the snapshots in a repo. It limits each snapshot to a single line.
+
+https://github.com/restic/restic/issues/1218
+https://github.com/restic/restic/pull/1223

changelog/0.7.2/issue-317

@@ -0,0 +1,10 @@
+Enhancement: Add `--exclude-caches` and `--exclude-if-present`
+
+A new option `--exclude-caches` was added that allows excluding cache
+directories (that are tagged as such). This is a special case of a more generic
+option `--exclude-if-present` which excludes a directory if a file with a
+specific name (and contents) is present.
+
+https://github.com/restic/restic/issues/317
+https://github.com/restic/restic/pull/1170
+https://github.com/restic/restic/pull/1224

changelog/0.7.2/issues-697

@@ -0,0 +1,4 @@
+Enhancement: Automatically generate man pages for all restic commands
+
+https://github.com/restic/restic/issues/697
+https://github.com/restic/restic/pull/1147

changelog/0.7.2/pull-1044

@@ -0,0 +1,3 @@
+Enhancement: Improve `restore`, do not traverse/load excluded directories
+
+https://github.com/restic/restic/pull/1044

changelog/0.7.2/pull-1061

@@ -0,0 +1,3 @@
+Enhancement: Add Dockerfile and official Docker image
+
+https://github.com/restic/restic/pull/1061

changelog/0.7.2/pull-1126

@@ -0,0 +1,7 @@
+Enhancement: Use the standard Go git repository layout, use `dep` for vendoring
+
+The git repository layout was changed to resemble the layout typically used in
+Go projects, we're not using `gb` for building restic any more and vendoring
+the dependencies is now taken care of by `dep`.
+
+https://github.com/restic/restic/pull/1126

changelog/0.7.2/pull-1134

@@ -0,0 +1,5 @@
+Enhancement: Add support for storing backups on Google Cloud Storage
+
+https://github.com/restic/restic/pull/1134
+https://github.com/restic/restic/pull/1052
+https://github.com/restic/restic/issues/211

changelog/0.7.2/pull-1144

@@ -0,0 +1,3 @@
+Enhancement: Properly report errors when reading files with exclude patterns.
+
+https://github.com/restic/restic/pull/1144

changelog/0.7.2/pull-1149

@@ -0,0 +1,8 @@
+Enhancement: Add support for storing backups on Microsoft Azure Blob Storage
+
+The library we're using to access the service requires Go 1.8, so restic now
+needs at least Go 1.8.
+
+https://github.com/restic/restic/pull/1149
+https://github.com/restic/restic/pull/1059
+https://github.com/restic/restic/issues/609

changelog/0.7.2/pull-1164

@@ -0,0 +1,3 @@
+Bugfix: Make the `key remove` command behave as documented
+
+https://github.com/restic/restic/pull/1164

changelog/0.7.2/pull-1191

@@ -0,0 +1,8 @@
+Bugfix: Make sure to write profiling files on interrupt
+
+Since a few releases restic had the ability to write profiling files for memory
+and CPU usage when `debug` is enabled. It was discovered that when restic is
+interrupted (ctrl+c is pressed), the proper shutdown hook is not run. This is
+now corrected.
+
+https://github.com/restic/restic/pull/1191

changelog/0.7.2/pull-1196

@@ -0,0 +1,3 @@
+Enhancement: Add `--group-by` to `forget` command for flexible grouping
+
+https://github.com/restic/restic/pull/1196

changelog/0.7.2/pull-1203

@@ -0,0 +1,5 @@
+Enhancement: Print stats on all BSD systems when SIGINFO (ctrl+t) is received
+
+https://github.com/restic/restic/pull/1203
+https://github.com/restic/restic/pull/1082#issuecomment-326279920
+

changelog/0.7.2/pull-1205

@@ -0,0 +1,3 @@
+Enhancement: Allow specifying time/date for a backup with `--time`
+
+https://github.com/restic/restic/pull/1205

changelog/0.7.3/issue-1246

@@ -0,0 +1,9 @@
+Bugfix: List all files stored in Google Cloud Storage
+
+For large backups stored in Google Cloud Storage, the `prune` command fails
+because listing only returns the first 1000 files. This has been corrected, no
+data is lost in the process. In addition, a plausibility check was added to
+`prune`.
+
+https://github.com/restic/restic/issues/1246
+https://github.com/restic/restic/pull/1247

changelog/0.8.0/issue-1102

@@ -0,0 +1,9 @@
+Enhancement: Add subdirectory `ids` to fuse mount
+
+The fuse mount now has an `ids` subdirectory which contains the snapshots below
+their (short) IDs.
+
+https://github.com/restic/restic/issues/1102
+https://github.com/restic/restic/pull/1299
+https://github.com/restic/restic/pull/1320
+

changelog/0.8.0/issue-1114

@@ -0,0 +1,10 @@
+Enhancement: Add `--cacert` to specify TLS certificates to check against
+
+We've added the `--cacert` option which can be used to pass one (or more) CA
+certificates to restic. These are used in addition to the system CA
+certificates to verify HTTPS certificates (e.g. for the REST backend).
+
+https://github.com/restic/restic/issues/1114
+https://github.com/restic/restic/pull/1276
+
+

changelog/0.8.0/issue-1216

@@ -0,0 +1,9 @@
+Enhancement: Add upload/download limiting
+
+We've added support for rate limiting through `--limit-upload` and
+`--limit-download` flags.
+
+https://github.com/restic/restic/issues/1216
+https://github.com/restic/restic/pull/1336
+https://github.com/restic/restic/pull/1358
+

changelog/0.8.0/issue-1256

@@ -0,0 +1,7 @@
+Bugfix: Re-enable workaround for S3 backend
+
+We've re-enabled a workaround for `minio-go` (the library we're using to
+access s3 backends), this reduces memory usage.
+
+https://github.com/restic/restic/issues/1256
+https://github.com/restic/restic/pull/1267

changelog/0.8.0/issue-1271

@@ -0,0 +1,6 @@
+Enhancement: Cache results for excludes for `backup`
+
+The `backup` command now caches the result of excludes for a directory.
+
+https://github.com/restic/restic/issues/1271
+https://github.com/restic/restic/pull/1326

changelog/0.8.0/issue-1274

@@ -0,0 +1,9 @@
+Enhancement: Add `generate` command, replaces `manpage` and `autocomplete`
+
+The `generate` command has been added, which replaces the now removed
+commands `manpage` and `autocomplete`. This release of restic contains the
+most recent manpages in `doc/man` and the auto-completion files for bash and
+zsh in `doc/bash-completion.sh` and `doc/zsh-completion.zsh`
+
+https://github.com/restic/restic/issues/1274
+https://github.com/restic/restic/pull/1282

changelog/0.8.0/issue-1291

@@ -0,0 +1,8 @@
+Bugfix: Reuse backend TCP connections to BackBlaze B2
+
+A bug was discovered in the library we're using to access Backblaze, it now
+reuses already established TCP connections which should be a lot faster and
+not cause network failures any more.
+
+https://github.com/restic/restic/issues/1291
+https://github.com/restic/restic/pull/1301

changelog/0.8.0/issue-1367

@@ -0,0 +1,7 @@
+Enhancement: Allow comments in files read from via `--file-from`
+
+When the list of files/dirs to be saved is read from a file with
+`--files-from`, comment lines (starting with `#`) are now ignored.
+
+https://github.com/restic/restic/issues/1367
+https://github.com/restic/restic/pull/1368

changelog/0.8.0/issue-1445

@@ -0,0 +1,18 @@
+Security: Prevent writing outside the target directory during restore
+
+A vulnerability was found in the restic restorer, which allowed attackers in
+special circumstances to restore files to a location outside of the target
+directory. Due to the circumstances we estimate this to be a low-risk
+vulnerability, but urge all users to upgrade to the latest version of restic.
+
+Exploiting the vulnerability requires a Linux/Unix system which saves backups
+via restic and a Windows systems which restores files from the repo. In
+addition, the attackers need to be able to create create files with arbitrary
+names which are then saved to the restic repo. For example, by creating a file
+named "..\test.txt" (which is a perfectly legal filename on Linux) and
+restoring a snapshot containing this file on Windows, it would be written to
+the parent of the target directory.
+
+We'd like to thank Tyler Spivey for reporting this responsibly!
+
+https://github.com/restic/restic/pull/1445

changelog/0.8.0/issue-448

@@ -0,0 +1,9 @@
+Enhancement: sftp backend prompts for password
+
+The sftp backend now prompts for the password if a password is necessary for
+login.
+
+https://github.com/restic/restic/issues/448
+https://github.com/restic/restic/pull/1270
+
+

changelog/0.8.0/issue-510

@@ -0,0 +1,7 @@
+Enhancement: Add `dump` command
+
+We've added the `dump` command which prints a file from a snapshot to
+stdout. This can e.g. be used to restore files read with `backup --stdin`.
+
+https://github.com/restic/restic/issues/510
+https://github.com/restic/restic/pull/1346

changelog/0.8.0/pull-1040

@@ -0,0 +1,23 @@
+Enhancement: Add local metadata cache
+
+We've added a local cache for metadata so that restic doesn't need to load
+all metadata (snapshots, indexes, ...) from the repo each time it starts. By
+default the cache is active, but there's a new global option `--no-cache`
+that can be used to disable the cache. By deafult, the cache a standard
+cache folder for the OS, which can be overridden with `--cache-dir`.  The
+cache will automatically populate, indexes and snapshots are saved as they
+are loaded. Cache directories for repos that haven't been used recently can
+automatically be removed by restic with the `--cleanup-cache` option.
+
+A related change was to by default create pack files in the repo that contain
+either data or metadata, not both mixed together. This allows easy caching of
+only the metadata files. The next run of `restic prune` will untangle mixed
+files automatically.
+
+https://github.com/restic/restic/pull/1040
+https://github.com/restic/restic/issues/29
+https://github.com/restic/restic/issues/738
+https://github.com/restic/restic/issues/282
+https://github.com/restic/restic/pull/1287
+https://github.com/restic/restic/pull/1436
+https://github.com/restic/restic/pull/1265

changelog/0.8.0/pull-1249

@@ -0,0 +1,6 @@
+Enhancement: Add `latest` symlink in fuse mount
+
+The directory structure in the fuse mount now exposes a symlink `latest`
+which points to the latest snapshot in that particular directory.
+
+https://github.com/restic/restic/pull/1249

changelog/0.8.0/pull-1269

@@ -0,0 +1,6 @@
+Enhancement: Add `--compact` to `forget` command
+
+The option `--compact` was added to the `forget` command to provide the same
+compact view as the `snapshots` command.
+
+https://github.com/restic/restic/pull/1269

changelog/0.8.0/pull-1281

@@ -0,0 +1,7 @@
+Enhancement: Google Cloud Storage backend needs less permissions
+
+The Google Cloud Storage backend no longer requires the service account to
+have the `storage.buckets.get` permission ("Storage Admin" role) in `restic
+init` if the bucket already exists.
+
+https://github.com/restic/restic/pull/1281

changelog/0.8.0/pull-1317

@@ -0,0 +1,7 @@
+Bugfix: Run prune when `forget --prune` is called with just snapshot IDs
+
+A bug in the `forget` command caused `prune` not to be run when `--prune` was
+specified without a policy, e.g. when only snapshot IDs that should be
+forgotten are listed manually.
+
+https://github.com/restic/restic/pull/1317

changelog/0.8.0/pull-1319

@@ -0,0 +1,8 @@
+Enhancement: Make `check` print `no errors found` explicitly
+
+The `check` command now explicetly prints `No errors were found` when no errors
+could be found.
+
+https://github.com/restic/restic/pull/1319
+https://github.com/restic/restic/issues/1303
+

changelog/0.8.0/pull-1353

@@ -0,0 +1,3 @@
+Enhancement: Retry failed backend requests
+
+https://github.com/restic/restic/pull/1353

changelog/0.8.0/pull-1437

@@ -0,0 +1,10 @@
+Bugfix: Remove implicit path `/restic` for the s3 backend
+
+The s3 backend used the subdir `restic` within a bucket if no explicit path
+after the bucket name was specified. Since this version, restic does not use
+this default path any more. If you created a repo on s3 in a bucket without
+specifying a path within the bucket, you need to add `/restic` at the end of
+the repository specification to access your repo: `s3:s3.amazonaws.com/bucket/restic`
+
+https://github.com/restic/restic/pull/1437
+https://github.com/restic/restic/issues/1292

changelog/0.8.1/issue-1457

@@ -0,0 +1,4 @@
+Bugfix: Improve s3 backend with DigitalOcean Spaces
+
+https://github.com/restic/restic/issues/1457
+https://github.com/restic/restic/pull/1459

changelog/0.8.1/pull-1436

@@ -0,0 +1,9 @@
+Enhancement: Add code to detect old cache directories
+
+We've added code to detect old cache directories of repositories that
+haven't been used in a long time, restic now prints a note when it detects
+that such dirs exist. Also, the option `--cleanup-cache` was added to
+automatically remove such directories. That's not a problem because the
+cache will be rebuild once a repo is accessed again.
+
+https://github.com/restic/restic/pull/1436

changelog/0.8.1/pull-1439

@@ -0,0 +1,6 @@
+Enhancement: Improve cancellation logic
+
+The cancellation logic was improved, restic can now shut down cleanly when
+requested to do so (e.g. via ctrl+c).
+
+https://github.com/restic/restic/pull/1439

changelog/0.8.1/pull-1452

@@ -0,0 +1,9 @@
+Change: Do not save atime by default
+
+By default, the access time for files and dirs is not saved any more. It is
+not possible to reliably disable updating the access time during a backup,
+so for the next backup the access time is different again. This means a lot
+of metadata is saved. If you want to save the access time anyway, pass
+`--with-atime` to the `backup` command.
+
+https://github.com/restic/restic/pull/1452

changelog/0.8.1/pull-1454

@@ -0,0 +1,6 @@
+Bugfix: Correct cache dir location for Windows and Darwin
+
+The cache directory on Windows and Darwin was not correct, instead the
+directory `.cache` was used.
+
+https://github.com/restic/restic/pull/1454

changelog/0.8.1/pull-1459

@@ -0,0 +1,9 @@
+Bugfix: Disable handling SIGPIPE
+
+We've disabled handling SIGPIPE again. Turns out, writing to broken TCP
+connections also raised SIGPIPE, so restic exits on the first write to a
+broken connection. Instead, restic should retry the request.
+
+https://github.com/restic/restic/pull/1459
+https://github.com/restic/restic/issues/1457
+https://github.com/restic/restic/issues/1466

changelog/0.8.1/pull-1462

@@ -0,0 +1,8 @@
+Enhancement: Add the `diff` command
+
+The command `diff` was added, it allows comparing two snapshots and listing
+all differences.
+
+https://github.com/restic/restic/issues/11
+https://github.com/restic/restic/issues/1460
+https://github.com/restic/restic/pull/1462

changelog/0.8.2/issue-1506

@@ -0,0 +1,4 @@
+Bugfix: Limit bandwith at the http.RoundTripper for HTTP based backends
+
+https://github.com/restic/restic/issues/1506
+https://github.com/restic/restic/pull/1511

changelog/0.8.2/issue-1512

@@ -0,0 +1,9 @@
+Bugfix: Restore directory permissions as the last step
+
+This change allows restoring into directories that were not writable during
+backup. Before, restic created the directory, set the read-only mode and then
+failed to create files in the directory. This change now restores the directory
+(with its permissions) as the very last step.
+
+https://github.com/restic/restic/issues/1512
+https://github.com/restic/restic/pull/1536

changelog/0.8.2/issue-1528

@@ -0,0 +1,4 @@
+Bugfix: Correctly create missing subdirs in data/
+
+https://github.com/restic/restic/issues/1528
+https://github.com/restic/restic/pull/1529

changelog/0.8.2/pull-1507

@@ -0,0 +1,3 @@
+Enhancement: Only reload snapshots once per minute for fuse mount
+
+https://github.com/restic/restic/pull/1507

changelog/0.8.2/pull-1538

@@ -0,0 +1,7 @@
+Enhancement: Reduce memory allocations for querying the index
+
+This change reduces the internal memory allocations when the index data
+structures in memory are queried if a blob (part of a file) already exists in
+the repo. It should speed up backup a bit, and maybe even reduce RAM usage.
+
+https://github.com/restic/restic/pull/1538

changelog/0.8.2/pull-1554

@@ -0,0 +1,7 @@
+Enhancement: fuse/mount: Correctly handle EOF, add template option
+
+We've added the `--snapshot-template` string, which can be used to specify a
+template for a snapshot directory. In addition, accessing data after the end of
+a file via the fuse mount is now handled correctly.
+
+https://github.com/restic/restic/pull/1554

changelog/CHANGELOG.tmpl

@@ -0,0 +1,32 @@
+{{- range $changes := . }}{{ with $changes -}}
+Changelog for restic {{ .Version }} ({{ .Date }})
+=======================================
+
+The following sections list the changes in restic {{ .Version }} relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+{{ range $entry := .Entries }}{{ with $entry }}
+ * {{ .TypeShort }} #{{ .PrimaryID }}: {{ .Title }}
+{{- end }}{{ end }}
+
+Details
+-------
+{{ range $entry := .Entries }}{{ with $entry }}
+ * {{ .Type }} #{{ .PrimaryID }}: {{ .Title }}
+{{ range $par := .Paragraphs }}
+   {{ wrap $par 80 3 }}
+{{ end -}}
+{{ range $id := .Issues }}
+   https://github.com/restic/restic/issues/{{ $id -}}
+{{ end -}}
+{{ range $id := .PRs }}
+   https://github.com/restic/restic/pull/{{ $id -}}
+{{ end -}}
+{{ range $url := .OtherURLs }}
+   {{ $url -}}
+{{ end }}
+{{ end }}{{ end }}
+
+{{ end }}{{ end -}}

changelog/changelog-entry.tmpl

@@ -0,0 +1,12 @@
+Bugfix: Fix behavior for foobar (in present tense)
+
+We've fixed the behavior for foobar, a long-standing annoyance for restic
+users.
+
+The text in the paragraphs is written in past tense. The last section is a list
+of issue URLs, PR URLs and other URLs. The first issue ID (or the first PR ID,
+in case there aren't any issue links) is used as the primary ID.
+
+https://github.com/restic/restic/issues/1234
+https://github.com/restic/restic/pull/55555
+https://forum.restic/.net/foo/bar/baz

changelog/changelog-github.tmpl

@@ -0,0 +1,32 @@
+{{- range $changes := . }}{{ with $changes -}}
+Changelog for restic {{ .Version }} ({{ .Date }})
+=======================================
+
+The following sections list the changes in restic {{ .Version }} relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+{{ range $entry := .Entries }}{{ with $entry }}
+ * {{ .TypeShort }} [#{{ .PrimaryID }}]({{ .PrimaryURL }}): {{ .Title }}
+{{- end }}{{ end }}
+
+Details
+-------
+{{ range $entry := .Entries }}{{ with $entry }}
+ * {{ .Type }} #{{ .PrimaryID }}: {{ .Title }}
+{{ range $par := .Paragraphs }}
+   {{ $par }}
+{{ end }}
+   {{ range $id := .Issues -}}
+{{ ` ` }}[#{{ $id }}](https://github.com/restic/restic/issues/{{ $id -}})
+{{- end -}}
+{{ range $id := .PRs -}}
+{{ ` ` }}[#{{ $id }}](https://github.com/restic/restic/pull/{{ $id -}})
+{{- end -}}
+{{ ` ` }}{{ range $url := .OtherURLs -}}
+{{ $url -}}
+{{- end }}
+{{ end }}{{ end }}
+
+{{ end }}{{ end -}}

changelog/releases

@@ -0,0 +1,14 @@
+# This file lists all versions for the changelog. Each line consists of the
+# version string, followed by an optional release date.
+# 
+# The resulting changelog generated by `calens` will list all versions in
+# exactly this order.
+0.8.2
+0.8.1 2017-12-27
+0.8.0 2017-11-26
+0.7.3 2017-09-20
+0.7.2 2017-09-13
+0.7.1 2017-07-22
+0.7.0 2017-07-01
+0.6.1 2017-06-01
+0.6.0 2017-05-29

cmd/restic/cleanup.go

@@ -25,16 +25,14 @@ func init() {
 	InstallSignalHandler()
 }
 
-// InstallSignalHandler listens for SIGINT and SIGPIPE, and triggers the cleanup handlers.
+// InstallSignalHandler listens for SIGINT, and triggers the cleanup handlers.
 func InstallSignalHandler() {
 	signal.Notify(cleanupHandlers.ch, syscall.SIGINT)
-	signal.Notify(cleanupHandlers.ch, syscall.SIGPIPE)
 }
 
-// SuspendSignalHandler removes the signal handler for SIGINT and SIGPIPE.
+// SuspendSignalHandler removes the signal handler for SIGINT.
 func SuspendSignalHandler() {
 	signal.Reset(syscall.SIGINT)
-	signal.Reset(syscall.SIGPIPE)
 }
 
 // AddCleanupHandler adds the function f to the list of cleanup handlers so
@@ -69,7 +67,7 @@ func RunCleanupHandlers() {
 	cleanupHandlers.list = nil
 }
 
-// CleanupHandler handles the SIGINT and SIGPIPE signals.
+// CleanupHandler handles the SIGINT signals.
 func CleanupHandler(c <-chan os.Signal) {
 	for s := range c {
 		debug.Log("signal %v received, cleaning up", s)

cmd/restic/cmd_backup.go

@@ -2,10 +2,10 @@ package main
 
 import (
 	"bufio"
-	"context"
 	"fmt"
 	"io"
 	"os"
+	"path"
 	"path/filepath"
 	"strings"
 	"time"
@@ -65,6 +65,7 @@ type BackupOptions struct {
 	Hostname         string
 	FilesFrom        string
 	TimeStamp        string
+	WithAtime        bool
 }
 
 var backupOptions BackupOptions
@@ -86,6 +87,7 @@ func init() {
 	f.StringVar(&backupOptions.Hostname, "hostname", "", "set the `hostname` for the snapshot manually. To prevent an expensive rescan use the \"parent\" flag")
 	f.StringVar(&backupOptions.FilesFrom, "files-from", "", "read the files to backup from file (can be combined with file args)")
 	f.StringVar(&backupOptions.TimeStamp, "time", "", "time of the backup (ex. '2012-11-01 22:08:41') (default: now)")
+	f.BoolVar(&backupOptions.WithAtime, "with-atime", false, "store the atime for all files and directories")
 }
 
 func newScanProgress(gopts GlobalOptions) *restic.Progress {
@@ -235,10 +237,16 @@ func readBackupFromStdin(opts BackupOptions, gopts GlobalOptions, args []string)
 		return errors.Fatal("when reading from stdin, no additional files can be specified")
 	}
 
-	if opts.StdinFilename == "" {
+	fn := opts.StdinFilename
+
+	if fn == "" {
 		return errors.Fatal("filename for backup from stdin must not be empty")
 	}
 
+	if filepath.Base(fn) != fn || path.Base(fn) != fn {
+		return errors.Fatal("filename is invalid (may not contain a directory, slash or backslash)")
+	}
+
 	if gopts.password == "" {
 		return errors.Fatal("unable to read password from stdin when data is to be read from stdin, use --password-file or $RESTIC_PASSWORD")
 	}
@@ -254,7 +262,7 @@ func readBackupFromStdin(opts BackupOptions, gopts GlobalOptions, args []string)
 		return err
 	}
 
-	err = repo.LoadIndex(context.TODO())
+	err = repo.LoadIndex(gopts.ctx)
 	if err != nil {
 		return err
 	}
@@ -265,7 +273,7 @@ func readBackupFromStdin(opts BackupOptions, gopts GlobalOptions, args []string)
 		Hostname:   opts.Hostname,
 	}
 
-	_, id, err := r.Archive(context.TODO(), opts.StdinFilename, os.Stdin, newArchiveStdinProgress(gopts))
+	_, id, err := r.Archive(gopts.ctx, fn, os.Stdin, newArchiveStdinProgress(gopts))
 	if err != nil {
 		return err
 	}
@@ -372,9 +380,8 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, args []string) error {
 		opts.ExcludeIfPresent = append(opts.ExcludeIfPresent, "CACHEDIR.TAG:Signature: 8a477f597d28d172789f06886806bc55")
 	}
 
-	rc := &rejectionCache{}
 	for _, spec := range opts.ExcludeIfPresent {
-		f, err := rejectIfPresent(spec, rc)
+		f, err := rejectIfPresent(spec)
 		if err != nil {
 			return err
 		}
@@ -403,7 +410,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, args []string) error {
 		rejectFuncs = append(rejectFuncs, f)
 	}
 
-	err = repo.LoadIndex(context.TODO())
+	err = repo.LoadIndex(gopts.ctx)
 	if err != nil {
 		return err
 	}
@@ -422,7 +429,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, args []string) error {
 
 	// Find last snapshot to set it as parent, if not already set
 	if !opts.Force && parentSnapshotID == nil {
-		id, err := restic.FindLatestSnapshot(context.TODO(), repo, target, []restic.TagList{}, opts.Hostname)
+		id, err := restic.FindLatestSnapshot(gopts.ctx, repo, target, []restic.TagList{}, opts.Hostname)
 		if err == nil {
 			parentSnapshotID = &id
 		} else if err != restic.ErrNoSnapshotFound {
@@ -453,6 +460,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, args []string) error {
 	arch := archiver.New(repo)
 	arch.Excludes = opts.Excludes
 	arch.SelectFilter = selectFilter
+	arch.WithAccessTime = opts.WithAtime
 
 	arch.Warn = func(dir string, fi os.FileInfo, err error) {
 		// TODO: make ignoring errors configurable
@@ -467,7 +475,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, args []string) error {
 		}
 	}
 
-	_, id, err := arch.Snapshot(context.TODO(), newArchiveProgress(gopts, stat), target, opts.Tags, opts.Hostname, parentSnapshotID, timeStamp)
+	_, id, err := arch.Snapshot(gopts.ctx, newArchiveProgress(gopts, stat), target, opts.Tags, opts.Hostname, parentSnapshotID, timeStamp)
 	if err != nil {
 		return err
 	}

cmd/restic/cmd_cat.go

@@ -1,7 +1,6 @@
 package main
 
 import (
-	"context"
 	"encoding/json"
 	"fmt"
 	"os"
@@ -75,7 +74,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 		fmt.Println(string(buf))
 		return nil
 	case "index":
-		buf, err := repo.LoadAndDecrypt(context.TODO(), restic.IndexFile, id)
+		buf, err := repo.LoadAndDecrypt(gopts.ctx, restic.IndexFile, id)
 		if err != nil {
 			return err
 		}
@@ -85,7 +84,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 
 	case "snapshot":
 		sn := &restic.Snapshot{}
-		err = repo.LoadJSONUnpacked(context.TODO(), restic.SnapshotFile, id, sn)
+		err = repo.LoadJSONUnpacked(gopts.ctx, restic.SnapshotFile, id, sn)
 		if err != nil {
 			return err
 		}
@@ -100,7 +99,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 		return nil
 	case "key":
 		h := restic.Handle{Type: restic.KeyFile, Name: id.String()}
-		buf, err := backend.LoadAll(context.TODO(), repo.Backend(), h)
+		buf, err := backend.LoadAll(gopts.ctx, repo.Backend(), h)
 		if err != nil {
 			return err
 		}
@@ -127,7 +126,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 		fmt.Println(string(buf))
 		return nil
 	case "lock":
-		lock, err := restic.LoadLock(context.TODO(), repo, id)
+		lock, err := restic.LoadLock(gopts.ctx, repo, id)
 		if err != nil {
 			return err
 		}
@@ -143,7 +142,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 	}
 
 	// load index, handle all the other types
-	err = repo.LoadIndex(context.TODO())
+	err = repo.LoadIndex(gopts.ctx)
 	if err != nil {
 		return err
 	}
@@ -151,7 +150,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 	switch tpe {
 	case "pack":
 		h := restic.Handle{Type: restic.DataFile, Name: id.String()}
-		buf, err := backend.LoadAll(context.TODO(), repo.Backend(), h)
+		buf, err := backend.LoadAll(gopts.ctx, repo.Backend(), h)
 		if err != nil {
 			return err
 		}
@@ -173,7 +172,7 @@ func runCat(gopts GlobalOptions, args []string) error {
 			blob := list[0]
 
 			buf := make([]byte, blob.Length)
-			n, err := repo.LoadBlob(context.TODO(), t, id, buf)
+			n, err := repo.LoadBlob(gopts.ctx, t, id, buf)
 			if err != nil {
 				return err
 			}

cmd/restic/cmd_check.go

@@ -1,7 +1,6 @@
 package main
 
 import (
-	"context"
 	"fmt"
 	"os"
 	"time"
@@ -104,7 +103,7 @@ func runCheck(opts CheckOptions, gopts GlobalOptions, args []string) error {
 	chkr := checker.New(repo)
 
 	Verbosef("load indexes\n")
-	hints, errs := chkr.LoadIndex(context.TODO())
+	hints, errs := chkr.LoadIndex(gopts.ctx)
 
 	dupFound := false
 	for _, hint := range hints {
@@ -129,7 +128,7 @@ func runCheck(opts CheckOptions, gopts GlobalOptions, args []string) error {
 	errChan := make(chan error)
 
 	Verbosef("check all packs\n")
-	go chkr.Packs(context.TODO(), errChan)
+	go chkr.Packs(gopts.ctx, errChan)
 
 	for err := range errChan {
 		errorsFound = true
@@ -138,7 +137,7 @@ func runCheck(opts CheckOptions, gopts GlobalOptions, args []string) error {
 
 	Verbosef("check snapshots, trees and blobs\n")
 	errChan = make(chan error)
-	go chkr.Structure(context.TODO(), errChan)
+	go chkr.Structure(gopts.ctx, errChan)
 
 	for err := range errChan {
 		errorsFound = true
@@ -165,7 +164,7 @@ func runCheck(opts CheckOptions, gopts GlobalOptions, args []string) error {
 		p := newReadProgress(gopts, restic.Stat{Blobs: chkr.CountPacks()})
 		errChan := make(chan error)
 
-		go chkr.ReadData(context.TODO(), p, errChan)
+		go chkr.ReadData(gopts.ctx, p, errChan)
 
 		for err := range errChan {
 			errorsFound = true

cmd/restic/cmd_debug.go

@@ -183,7 +183,7 @@ func runDebugDump(gopts GlobalOptions, args []string) error {
 		}
 	}
 
-	err = repo.LoadIndex(context.TODO())
+	err = repo.LoadIndex(gopts.ctx)
 	if err != nil {
 		return err
 	}

cmd/restic/cmd_diff.go

@@ -0,0 +1,356 @@
+package main
+
+import (
+	"context"
+	"path"
+	"reflect"
+	"sort"
+
+	"github.com/restic/restic/internal/debug"
+	"github.com/restic/restic/internal/errors"
+	"github.com/restic/restic/internal/repository"
+	"github.com/restic/restic/internal/restic"
+	"github.com/spf13/cobra"
+)
+
+var cmdDiff = &cobra.Command{
+	Use:   "diff snapshot-ID snapshot-ID",
+	Short: "Show differences between two snapshots",
+	Long: `
+The "diff" command shows differences from the first to the second snapshot. The
+first characters in each line display what has happened to a particular file or
+directory:
+
+ +  The item was added
+ -  The item was removed
+ U  The metadata (access mode, timestamps, ...) for the item was updated
+ M  The file's content was modified
+ T  The type was changed, e.g. a file was made a symlink
+`,
+	DisableAutoGenTag: true,
+	RunE: func(cmd *cobra.Command, args []string) error {
+		return runDiff(diffOptions, globalOptions, args)
+	},
+}
+
+// DiffOptions collects all options for the diff command.
+type DiffOptions struct {
+	ShowMetadata bool
+}
+
+var diffOptions DiffOptions
+
+func init() {
+	cmdRoot.AddCommand(cmdDiff)
+
+	f := cmdDiff.Flags()
+	f.BoolVar(&diffOptions.ShowMetadata, "metadata", false, "print changes in metadata")
+}
+
+func loadSnapshot(ctx context.Context, repo *repository.Repository, desc string) (*restic.Snapshot, error) {
+	id, err := restic.FindSnapshot(repo, desc)
+	if err != nil {
+		return nil, err
+	}
+
+	return restic.LoadSnapshot(ctx, repo, id)
+}
+
+// Comparer collects all things needed to compare two snapshots.
+type Comparer struct {
+	repo restic.Repository
+	opts DiffOptions
+}
+
+// DiffStat collects stats for all types of items.
+type DiffStat struct {
+	Files, Dirs, Others  int
+	DataBlobs, TreeBlobs int
+	Bytes                int
+}
+
+// Add adds stats information for node to s.
+func (s *DiffStat) Add(node *restic.Node) {
+	if node == nil {
+		return
+	}
+
+	switch node.Type {
+	case "file":
+		s.Files++
+	case "dir":
+		s.Dirs++
+	default:
+		s.Others++
+	}
+}
+
+// addBlobs adds the blobs of node to s.
+func addBlobs(bs restic.BlobSet, node *restic.Node) {
+	if node == nil {
+		return
+	}
+
+	switch node.Type {
+	case "file":
+		for _, blob := range node.Content {
+			h := restic.BlobHandle{
+				ID:   blob,
+				Type: restic.DataBlob,
+			}
+			bs.Insert(h)
+		}
+	case "dir":
+		h := restic.BlobHandle{
+			ID:   *node.Subtree,
+			Type: restic.TreeBlob,
+		}
+		bs.Insert(h)
+	}
+}
+
+// DiffStats collects the differences between two snapshots.
+type DiffStats struct {
+	ChangedFiles            int
+	Added                   DiffStat
+	Removed                 DiffStat
+	BlobsBefore, BlobsAfter restic.BlobSet
+}
+
+// NewDiffStats creates new stats for a diff run.
+func NewDiffStats() *DiffStats {
+	return &DiffStats{
+		BlobsBefore: restic.NewBlobSet(),
+		BlobsAfter:  restic.NewBlobSet(),
+	}
+}
+
+// updateBlobs updates the blob counters in the stats struct.
+func updateBlobs(repo restic.Repository, blobs restic.BlobSet, stats *DiffStat) {
+	for h := range blobs {
+		switch h.Type {
+		case restic.DataBlob:
+			stats.DataBlobs++
+		case restic.TreeBlob:
+			stats.TreeBlobs++
+		}
+
+		size, err := repo.LookupBlobSize(h.ID, h.Type)
+		if err != nil {
+			Warnf("unable to find blob size for %v: %v\n", h, err)
+			continue
+		}
+
+		stats.Bytes += int(size)
+	}
+}
+
+func (c *Comparer) printDir(ctx context.Context, mode string, stats *DiffStat, blobs restic.BlobSet, prefix string, id restic.ID) error {
+	debug.Log("print %v tree %v", mode, id)
+	tree, err := c.repo.LoadTree(ctx, id)
+	if err != nil {
+		return err
+	}
+
+	for _, node := range tree.Nodes {
+		name := path.Join(prefix, node.Name)
+		if node.Type == "dir" {
+			name += "/"
+		}
+		Printf("%-5s%v\n", mode, name)
+		stats.Add(node)
+		addBlobs(blobs, node)
+
+		if node.Type == "dir" {
+			err := c.printDir(ctx, mode, stats, blobs, name, *node.Subtree)
+			if err != nil {
+				Warnf("error: %v\n", err)
+			}
+		}
+	}
+
+	return nil
+}
+
+func uniqueNodeNames(tree1, tree2 *restic.Tree) (tree1Nodes, tree2Nodes map[string]*restic.Node, uniqueNames []string) {
+	names := make(map[string]struct{})
+	tree1Nodes = make(map[string]*restic.Node)
+	for _, node := range tree1.Nodes {
+		tree1Nodes[node.Name] = node
+		names[node.Name] = struct{}{}
+	}
+
+	tree2Nodes = make(map[string]*restic.Node)
+	for _, node := range tree2.Nodes {
+		tree2Nodes[node.Name] = node
+		names[node.Name] = struct{}{}
+	}
+
+	uniqueNames = make([]string, 0, len(names))
+	for name := range names {
+		uniqueNames = append(uniqueNames, name)
+	}
+
+	sort.Sort(sort.StringSlice(uniqueNames))
+	return tree1Nodes, tree2Nodes, uniqueNames
+}
+
+func (c *Comparer) diffTree(ctx context.Context, stats *DiffStats, prefix string, id1, id2 restic.ID) error {
+	debug.Log("diffing %v to %v", id1, id2)
+	tree1, err := c.repo.LoadTree(ctx, id1)
+	if err != nil {
+		return err
+	}
+
+	tree2, err := c.repo.LoadTree(ctx, id2)
+	if err != nil {
+		return err
+	}
+
+	tree1Nodes, tree2Nodes, names := uniqueNodeNames(tree1, tree2)
+
+	for _, name := range names {
+		node1, t1 := tree1Nodes[name]
+		node2, t2 := tree2Nodes[name]
+
+		addBlobs(stats.BlobsBefore, node1)
+		addBlobs(stats.BlobsAfter, node2)
+
+		switch {
+		case t1 && t2:
+			name := path.Join(prefix, name)
+			mod := ""
+
+			if node1.Type != node2.Type {
+				mod += "T"
+			}
+
+			if node2.Type == "dir" {
+				name += "/"
+			}
+
+			if node1.Type == "file" &&
+				node2.Type == "file" &&
+				!reflect.DeepEqual(node1.Content, node2.Content) {
+				mod += "M"
+				stats.ChangedFiles++
+			} else if c.opts.ShowMetadata && !node1.Equals(*node2) {
+				mod += "U"
+			}
+
+			if mod != "" {
+				Printf("%-5s%v\n", mod, name)
+			}
+
+			if node1.Type == "dir" && node2.Type == "dir" {
+				err := c.diffTree(ctx, stats, name, *node1.Subtree, *node2.Subtree)
+				if err != nil {
+					Warnf("error: %v\n", err)
+				}
+			}
+		case t1 && !t2:
+			prefix := path.Join(prefix, name)
+			if node1.Type == "dir" {
+				prefix += "/"
+			}
+			Printf("%-5s%v\n", "-", prefix)
+			stats.Removed.Add(node1)
+
+			if node1.Type == "dir" {
+				err := c.printDir(ctx, "-", &stats.Removed, stats.BlobsBefore, prefix, *node1.Subtree)
+				if err != nil {
+					Warnf("error: %v\n", err)
+				}
+			}
+		case !t1 && t2:
+			prefix := path.Join(prefix, name)
+			if node2.Type == "dir" {
+				prefix += "/"
+			}
+			Printf("%-5s%v\n", "+", prefix)
+			stats.Added.Add(node2)
+
+			if node2.Type == "dir" {
+				err := c.printDir(ctx, "+", &stats.Added, stats.BlobsAfter, prefix, *node2.Subtree)
+				if err != nil {
+					Warnf("error: %v\n", err)
+				}
+			}
+		}
+	}
+
+	return nil
+}
+
+func runDiff(opts DiffOptions, gopts GlobalOptions, args []string) error {
+	if len(args) != 2 {
+		return errors.Fatalf("specify two snapshot IDs")
+	}
+
+	ctx, cancel := context.WithCancel(gopts.ctx)
+	defer cancel()
+
+	repo, err := OpenRepository(gopts)
+	if err != nil {
+		return err
+	}
+
+	if err = repo.LoadIndex(ctx); err != nil {
+		return err
+	}
+
+	if !gopts.NoLock {
+		lock, err := lockRepo(repo)
+		defer unlockRepo(lock)
+		if err != nil {
+			return err
+		}
+	}
+
+	sn1, err := loadSnapshot(ctx, repo, args[0])
+	if err != nil {
+		return err
+	}
+
+	sn2, err := loadSnapshot(ctx, repo, args[1])
+	if err != nil {
+		return err
+	}
+
+	Verbosef("comparing snapshot %v to %v:\n\n", sn1.ID().Str(), sn2.ID().Str())
+
+	if sn1.Tree == nil {
+		return errors.Errorf("snapshot %v has nil tree", sn1.ID().Str())
+	}
+
+	if sn2.Tree == nil {
+		return errors.Errorf("snapshot %v has nil tree", sn2.ID().Str())
+	}
+
+	c := &Comparer{
+		repo: repo,
+		opts: diffOptions,
+	}
+
+	stats := NewDiffStats()
+
+	err = c.diffTree(ctx, stats, "/", *sn1.Tree, *sn2.Tree)
+	if err != nil {
+		return err
+	}
+
+	both := stats.BlobsBefore.Intersect(stats.BlobsAfter)
+	updateBlobs(repo, stats.BlobsBefore.Sub(both), &stats.Removed)
+	updateBlobs(repo, stats.BlobsAfter.Sub(both), &stats.Added)
+
+	Printf("\n")
+	Printf("Files:       %5d new, %5d removed, %5d changed\n", stats.Added.Files, stats.Removed.Files, stats.ChangedFiles)
+	Printf("Dirs:        %5d new, %5d removed\n", stats.Added.Dirs, stats.Removed.Dirs)
+	Printf("Others:      %5d new, %5d removed\n", stats.Added.Others, stats.Removed.Others)
+	Printf("Data Blobs:  %5d new, %5d removed\n", stats.Added.DataBlobs, stats.Removed.DataBlobs)
+	Printf("Tree Blobs:  %5d new, %5d removed\n", stats.Added.TreeBlobs, stats.Removed.TreeBlobs)
+	Printf("  Added:   %-5s\n", formatBytes(uint64(stats.Added.Bytes)))
+	Printf("  Removed: %-5s\n", formatBytes(uint64(stats.Removed.Bytes)))
+
+	return nil
+}

cmd/restic/cmd_dump.go

@@ -162,7 +162,7 @@ func runDump(opts DumpOptions, gopts GlobalOptions, args []string) error {
 		}
 	}
 
-	sn, err := restic.LoadSnapshot(context.TODO(), repo, id)
+	sn, err := restic.LoadSnapshot(gopts.ctx, repo, id)
 	if err != nil {
 		Exitf(2, "loading snapshot %q failed: %v", snapshotIDString, err)
 	}

cmd/restic/cmd_find.go

@@ -180,7 +180,7 @@ type Finder struct {
 	notfound restic.IDSet
 }
 
-func (f *Finder) findInTree(treeID restic.ID, prefix string) error {
+func (f *Finder) findInTree(ctx context.Context, treeID restic.ID, prefix string) error {
 	if f.notfound.Has(treeID) {
 		debug.Log("%v skipping tree %v, has already been checked", prefix, treeID.Str())
 		return nil
@@ -188,7 +188,7 @@ func (f *Finder) findInTree(treeID restic.ID, prefix string) error {
 
 	debug.Log("%v checking tree %v\n", prefix, treeID.Str())
 
-	tree, err := f.repo.LoadTree(context.TODO(), treeID)
+	tree, err := f.repo.LoadTree(ctx, treeID)
 	if err != nil {
 		return err
 	}
@@ -224,7 +224,7 @@ func (f *Finder) findInTree(treeID restic.ID, prefix string) error {
 		}
 
 		if node.Type == "dir" {
-			if err := f.findInTree(*node.Subtree, filepath.Join(prefix, node.Name)); err != nil {
+			if err := f.findInTree(ctx, *node.Subtree, filepath.Join(prefix, node.Name)); err != nil {
 				return err
 			}
 		}
@@ -237,14 +237,11 @@ func (f *Finder) findInTree(treeID restic.ID, prefix string) error {
 	return nil
 }
 
-func (f *Finder) findInSnapshot(sn *restic.Snapshot) error {
+func (f *Finder) findInSnapshot(ctx context.Context, sn *restic.Snapshot) error {
 	debug.Log("searching in snapshot %s\n  for entries within [%s %s]", sn.ID(), f.pat.oldest, f.pat.newest)
 
 	f.out.newsn = sn
-	if err := f.findInTree(*sn.Tree, string(filepath.Separator)); err != nil {
+	return f.findInTree(ctx, *sn.Tree, string(filepath.Separator))
-		return err
-	}
-	return nil
 }
 
 func runFind(opts FindOptions, gopts GlobalOptions, args []string) error {
@@ -284,7 +281,7 @@ func runFind(opts FindOptions, gopts GlobalOptions, args []string) error {
 		}
 	}
 
-	if err = repo.LoadIndex(context.TODO()); err != nil {
+	if err = repo.LoadIndex(gopts.ctx); err != nil {
 		return err
 	}
 
@@ -298,7 +295,7 @@ func runFind(opts FindOptions, gopts GlobalOptions, args []string) error {
 		notfound: restic.NewIDSet(),
 	}
 	for sn := range FindFilteredSnapshots(ctx, repo, opts.Host, opts.Tags, opts.Paths, opts.Snapshots) {
-		if err = f.findInSnapshot(sn); err != nil {
+		if err = f.findInSnapshot(ctx, sn); err != nil {
 			return err
 		}
 	}

cmd/restic/cmd_forget.go

@@ -125,7 +125,7 @@ func runForget(opts ForgetOptions, gopts GlobalOptions, args []string) error {
 			// When explicit snapshots args are given, remove them immediately.
 			if !opts.DryRun {
 				h := restic.Handle{Type: restic.SnapshotFile, Name: sn.ID().String()}
-				if err = repo.Backend().Remove(context.TODO(), h); err != nil {
+				if err = repo.Backend().Remove(gopts.ctx, h); err != nil {
 					return err
 				}
 				Verbosef("removed snapshot %v\n", sn.ID().Str())
@@ -162,9 +162,6 @@ func runForget(opts ForgetOptions, gopts GlobalOptions, args []string) error {
 			snapshotGroups[string(k)] = append(snapshotGroups[string(k)], sn)
 		}
 	}
-	if len(args) > 0 {
-		return nil
-	}
 
 	policy := restic.ExpirePolicy{
 		Last:    opts.Last,
@@ -176,11 +173,11 @@ func runForget(opts ForgetOptions, gopts GlobalOptions, args []string) error {
 		Tags:    opts.KeepTags,
 	}
 
-	if policy.Empty() {
+	if policy.Empty() && len(args) == 0 {
 		Verbosef("no policy was specified, no snapshots will be removed\n")
-		return nil
 	}
 
+	if !policy.Empty() {
 		for k, snapshotGroup := range snapshotGroups {
 			var key key
 			if json.Unmarshal([]byte(k), &key) != nil {
@@ -223,13 +220,14 @@ func runForget(opts ForgetOptions, gopts GlobalOptions, args []string) error {
 			if !opts.DryRun {
 				for _, sn := range remove {
 					h := restic.Handle{Type: restic.SnapshotFile, Name: sn.ID().String()}
-				err = repo.Backend().Remove(context.TODO(), h)
+					err = repo.Backend().Remove(gopts.ctx, h)
 					if err != nil {
 						return err
 					}
 				}
 			}
 		}
+	}
 
 	if removeSnapshots > 0 && opts.Prune {
 		Verbosef("%d snapshots have been removed, running prune\n", removeSnapshots)

cmd/restic/cmd_init.go

@@ -1,8 +1,6 @@
 package main
 
 import (
-	"context"
-
 	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/repository"
 
@@ -44,7 +42,7 @@ func runInit(gopts GlobalOptions, args []string) error {
 
 	s := repository.New(be)
 
-	err = s.Init(context.TODO(), gopts.password)
+	err = s.Init(gopts.ctx, gopts.password)
 	if err != nil {
 		return errors.Fatalf("create key in backend at %s failed: %v\n", gopts.Repo, err)
 	}

cmd/restic/cmd_key.go

@@ -76,7 +76,7 @@ func addKey(gopts GlobalOptions, repo *repository.Repository) error {
 		return err
 	}
 
-	id, err := repository.AddKey(context.TODO(), repo, pw, repo.Key())
+	id, err := repository.AddKey(gopts.ctx, repo, pw, repo.Key())
 	if err != nil {
 		return errors.Fatalf("creating new key failed: %v\n", err)
 	}
@@ -86,13 +86,13 @@ func addKey(gopts GlobalOptions, repo *repository.Repository) error {
 	return nil
 }
 
-func deleteKey(repo *repository.Repository, name string) error {
+func deleteKey(ctx context.Context, repo *repository.Repository, name string) error {
 	if name == repo.KeyName() {
 		return errors.Fatal("refusing to remove key currently used to access repository")
 	}
 
 	h := restic.Handle{Type: restic.KeyFile, Name: name}
-	err := repo.Backend().Remove(context.TODO(), h)
+	err := repo.Backend().Remove(ctx, h)
 	if err != nil {
 		return err
 	}
@@ -107,13 +107,13 @@ func changePassword(gopts GlobalOptions, repo *repository.Repository) error {
 		return err
 	}
 
-	id, err := repository.AddKey(context.TODO(), repo, pw, repo.Key())
+	id, err := repository.AddKey(gopts.ctx, repo, pw, repo.Key())
 	if err != nil {
 		return errors.Fatalf("creating new key failed: %v\n", err)
 	}
 
 	h := restic.Handle{Type: restic.KeyFile, Name: repo.KeyName()}
-	err = repo.Backend().Remove(context.TODO(), h)
+	err = repo.Backend().Remove(gopts.ctx, h)
 	if err != nil {
 		return err
 	}
@@ -165,7 +165,7 @@ func runKey(gopts GlobalOptions, args []string) error {
 			return err
 		}
 
-		return deleteKey(repo, id)
+		return deleteKey(gopts.ctx, repo, id)
 	case "passwd":
 		lock, err := lockRepoExclusive(repo)
 		defer unlockRepo(lock)

cmd/restic/cmd_list.go

@@ -1,7 +1,6 @@
 package main
 
 import (
-	"context"
 	"fmt"
 
 	"github.com/restic/restic/internal/errors"
@@ -58,7 +57,7 @@ func runList(opts GlobalOptions, args []string) error {
 	case "locks":
 		t = restic.LockFile
 	case "blobs":
-		idx, err := index.Load(context.TODO(), repo, nil)
+		idx, err := index.Load(opts.ctx, repo, nil)
 		if err != nil {
 			return err
 		}
@@ -74,7 +73,7 @@ func runList(opts GlobalOptions, args []string) error {
 		return errors.Fatal("invalid type")
 	}
 
-	for id := range repo.List(context.TODO(), t) {
+	for id := range repo.List(opts.ctx, t) {
 		Printf("%s\n", id)
 	}
 

cmd/restic/cmd_ls.go

@@ -46,8 +46,8 @@ func init() {
 	flags.StringArrayVar(&lsOptions.Paths, "path", nil, "only consider snapshots which include this (absolute) `path`, when no snapshot ID is given")
 }
 
-func printTree(repo *repository.Repository, id *restic.ID, prefix string) error {
+func printTree(ctx context.Context, repo *repository.Repository, id *restic.ID, prefix string) error {
-	tree, err := repo.LoadTree(context.TODO(), *id)
+	tree, err := repo.LoadTree(ctx, *id)
 	if err != nil {
 		return err
 	}
@@ -56,7 +56,7 @@ func printTree(repo *repository.Repository, id *restic.ID, prefix string) error
 		Printf("%s\n", formatNode(prefix, entry, lsOptions.ListLong))
 
 		if entry.Type == "dir" && entry.Subtree != nil {
-			if err = printTree(repo, entry.Subtree, filepath.Join(prefix, entry.Name)); err != nil {
+			if err = printTree(ctx, repo, entry.Subtree, filepath.Join(prefix, entry.Name)); err != nil {
 				return err
 			}
 		}
@@ -75,7 +75,7 @@ func runLs(opts LsOptions, gopts GlobalOptions, args []string) error {
 		return err
 	}
 
-	if err = repo.LoadIndex(context.TODO()); err != nil {
+	if err = repo.LoadIndex(gopts.ctx); err != nil {
 		return err
 	}
 
@@ -84,7 +84,7 @@ func runLs(opts LsOptions, gopts GlobalOptions, args []string) error {
 	for sn := range FindFilteredSnapshots(ctx, repo, opts.Host, opts.Tags, opts.Paths, args) {
 		Verbosef("snapshot %s of %v at %s):\n", sn.ID().Str(), sn.Paths, sn.Time)
 
-		if err = printTree(repo, sn.Tree, string(filepath.Separator)); err != nil {
+		if err = printTree(gopts.ctx, repo, sn.Tree, string(filepath.Separator)); err != nil {
 			return err
 		}
 	}

cmd/restic/cmd_mount.go

@@ -4,8 +4,9 @@
 package main
 
 import (
-	"context"
 	"os"
+	"strings"
+	"time"
 
 	"github.com/spf13/cobra"
 
@@ -26,6 +27,21 @@ var cmdMount = &cobra.Command{
 	Long: `
 The "mount" command mounts the repository via fuse to a directory. This is a
 read-only mount.
+
+Snapshot Directories
+====================
+
+If you need a different template for all directories that contain snapshots,
+you can pass a template via --snapshot-template. Example without colons:
+
+    --snapshot-template "2006-01-02_15-04-05"
+
+You need to specify a sample format for exactly the following timestamp:
+
+    Mon Jan 2 15:04:05 -0700 MST 2006
+
+For details please see the documentation for time.Format() at:
+  https://godoc.org/time#Time.Format
 `,
 	DisableAutoGenTag: true,
 	RunE: func(cmd *cobra.Command, args []string) error {
@@ -41,6 +57,7 @@ type MountOptions struct {
 	Host             string
 	Tags             restic.TagLists
 	Paths            []string
+	SnapshotTemplate string
 }
 
 var mountOptions MountOptions
@@ -56,6 +73,8 @@ func init() {
 	mountFlags.StringVarP(&mountOptions.Host, "host", "H", "", `only consider snapshots for this host`)
 	mountFlags.Var(&mountOptions.Tags, "tag", "only consider snapshots which include this `taglist`")
 	mountFlags.StringArrayVar(&mountOptions.Paths, "path", nil, "only consider snapshots which include this (absolute) `path`")
+
+	mountFlags.StringVar(&mountOptions.SnapshotTemplate, "snapshot-template", time.RFC3339, "set `template` to use for snapshot dirs")
 }
 
 func mount(opts MountOptions, gopts GlobalOptions, mountpoint string) error {
@@ -73,7 +92,7 @@ func mount(opts MountOptions, gopts GlobalOptions, mountpoint string) error {
 		return err
 	}
 
-	err = repo.LoadIndex(context.TODO())
+	err = repo.LoadIndex(gopts.ctx)
 	if err != nil {
 		return err
 	}
@@ -113,8 +132,9 @@ func mount(opts MountOptions, gopts GlobalOptions, mountpoint string) error {
 		Host:             opts.Host,
 		Tags:             opts.Tags,
 		Paths:            opts.Paths,
+		SnapshotTemplate: opts.SnapshotTemplate,
 	}
-	root, err := fuse.NewRoot(context.TODO(), repo, cfg)
+	root, err := fuse.NewRoot(gopts.ctx, repo, cfg)
 	if err != nil {
 		return err
 	}
@@ -137,6 +157,10 @@ func umount(mountpoint string) error {
 }
 
 func runMount(opts MountOptions, gopts GlobalOptions, args []string) error {
+	if strings.ContainsAny(opts.SnapshotTemplate, `\/`) {
+		return errors.Fatal("snapshot template string contains a slash (/) or backslash (\\) character")
+	}
+
 	if len(args) == 0 {
 		return errors.Fatal("wrong number of parameters")
 	}

cmd/restic/cmd_rebuild_index.go

@@ -3,6 +3,7 @@ package main
 import (
 	"context"
 
+	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/index"
 	"github.com/restic/restic/internal/restic"
 
@@ -66,7 +67,7 @@ func rebuildIndex(ctx context.Context, repo restic.Repository, ignorePacks resti
 
 	id, err := idx.Save(ctx, repo, supersedes)
 	if err != nil {
-		return err
+		return errors.Fatalf("unable to save index, last error was: %v", err)
 	}
 
 	Verbosef("saved new index as %v\n", id.Str())

cmd/restic/cmd_restore.go

@@ -53,8 +53,11 @@ func init() {
 func runRestore(opts RestoreOptions, gopts GlobalOptions, args []string) error {
 	ctx := gopts.ctx
 
-	if len(args) != 1 {
+	switch {
+	case len(args) == 0:
 		return errors.Fatal("no snapshot ID specified")
+	case len(args) > 1:
+		return errors.Fatalf("more than one snapshot ID specified: %v", args)
 	}
 
 	if opts.Target == "" {

cmd/restic/cmd_tag.go

@@ -53,7 +53,7 @@ func init() {
 	tagFlags.StringArrayVar(&tagOptions.Paths, "path", nil, "only consider snapshots which include this (absolute) `path`, when no snapshot-ID is given")
 }
 
-func changeTags(repo *repository.Repository, sn *restic.Snapshot, setTags, addTags, removeTags []string) (bool, error) {
+func changeTags(ctx context.Context, repo *repository.Repository, sn *restic.Snapshot, setTags, addTags, removeTags []string) (bool, error) {
 	var changed bool
 
 	if len(setTags) != 0 {
@@ -77,20 +77,20 @@ func changeTags(repo *repository.Repository, sn *restic.Snapshot, setTags, addTa
 		}
 
 		// Save the new snapshot.
-		id, err := repo.SaveJSONUnpacked(context.TODO(), restic.SnapshotFile, sn)
+		id, err := repo.SaveJSONUnpacked(ctx, restic.SnapshotFile, sn)
 		if err != nil {
 			return false, err
 		}
 
 		debug.Log("new snapshot saved as %v", id.Str())
 
-		if err = repo.Flush(); err != nil {
+		if err = repo.Flush(ctx); err != nil {
 			return false, err
 		}
 
 		// Remove the old snapshot.
 		h := restic.Handle{Type: restic.SnapshotFile, Name: sn.ID().String()}
-		if err = repo.Backend().Remove(context.TODO(), h); err != nil {
+		if err = repo.Backend().Remove(ctx, h); err != nil {
 			return false, err
 		}
 
@@ -125,7 +125,7 @@ func runTag(opts TagOptions, gopts GlobalOptions, args []string) error {
 	ctx, cancel := context.WithCancel(gopts.ctx)
 	defer cancel()
 	for sn := range FindFilteredSnapshots(ctx, repo, opts.Host, opts.Tags, opts.Paths, args) {
-		changed, err := changeTags(repo, sn, opts.SetTags, opts.AddTags, opts.RemoveTags)
+		changed, err := changeTags(ctx, repo, sn, opts.SetTags, opts.AddTags, opts.RemoveTags)
 		if err != nil {
 			Warnf("unable to modify the tags for snapshot ID %q, ignoring: %v\n", sn.ID(), err)
 			continue