Add version for 0.9.4

Update manpages and auto-completion
Generate CHANGELOG.md for 0.9.4
2026-02-23 01:06:23 +00:00 · 2019-01-06 21:40:32 +01:00 · 2019-01-06 21:40:32 +01:00 · 2019-01-06 21:40:31 +01:00 · 2019-01-06 21:39:56 +01:00 · 2019-01-06 21:39:56 +01:00
120 changed files with 5695 additions and 586 deletions
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -1,3 +1,131 @@
+Changelog for restic 0.9.4 (2019-01-06)
+=======================================
+
+The following sections list the changes in restic 0.9.4 relevant to
+restic users. The changes are ordered by importance.
+
+Summary
+-------
+
+ * Fix #1989: Google Cloud Storage: Respect bandwidth limit
+ * Fix #2040: Add host name filter shorthand flag for `stats` command
+ * Fix #2068: Correctly return error loading data
+ * Fix #2095: Consistently use local time for snapshots times
+ * Enh #1605: Concurrent restore
+ * Enh #2089: Increase granularity of the "keep within" retention policy
+ * Enh #2097: Add key hinting
+ * Enh #2017: Mount: Enforce FUSE Unix permissions with allow-other
+ * Enh #2070: Make all commands display timestamps in local time
+ * Enh #2085: Allow --files-from to be specified multiple times
+ * Enh #2094: Run command to get password
+
+Details
+-------
+
+ * Bugfix #1989: Google Cloud Storage: Respect bandwidth limit
+
+   The GCS backend did not respect the bandwidth limit configured, a previous commit
+   accidentally removed support for it.
+
+   https://github.com/restic/restic/issues/1989
+   https://github.com/restic/restic/pull/2100
+
+ * Bugfix #2040: Add host name filter shorthand flag for `stats` command
+
+   The default value for `--host` flag was set to 'H' (the shorthand version of the flag), this
+   caused the lookup for the latest snapshot to fail.
+
+   Add shorthand flag `-H` for `--host` (with empty default so if these flags are not specified the
+   latest snapshot will not filter by host name).
+
+   Also add shorthand `-H` for `backup` command.
+
+   https://github.com/restic/restic/issues/2040
+
+ * Bugfix #2068: Correctly return error loading data
+
+   In one case during `prune` and `check`, an error loading data from the backend is not returned
+   properly. This is now corrected.
+
+   https://github.com/restic/restic/issues/1999#issuecomment-433737921
+   https://github.com/restic/restic/pull/2068
+
+ * Bugfix #2095: Consistently use local time for snapshots times
+
+   By default snapshots created with restic backup were set to local time, but when the --time flag
+   was used the provided timestamp was parsed as UTC. With this change all snapshots times are set
+   to local time.
+
+   https://github.com/restic/restic/pull/2095
+
+ * Enhancement #1605: Concurrent restore
+
+   This change significantly improves restore performance, especially when using
+   high-latency remote repositories like B2.
+
+   The implementation now uses several concurrent threads to download and process multiple
+   remote files concurrently. To further reduce restore time, each remote file is downloaded
+   using a single repository request.
+
+   https://github.com/restic/restic/issues/1605
+   https://github.com/restic/restic/pull/1719
+
+ * Enhancement #2089: Increase granularity of the "keep within" retention policy
+
+   The `keep-within` option of the `forget` command now accepts time ranges with an hourly
+   granularity. For example, running `restic forget --keep-within 3d12h` will keep all the
+   snapshots made within three days and twelve hours from the time of the latest snapshot.
+
+   https://github.com/restic/restic/issues/2089
+   https://github.com/restic/restic/pull/2090
+
+ * Enhancement #2097: Add key hinting
+
+   Added a new option `--key-hint` and corresponding environment variable `RESTIC_KEY_HINT`.
+   The key hint is a key ID to try decrypting first, before other keys in the repository.
+
+   This change will benefit repositories with many keys; if the correct key hint is supplied then
+   restic only needs to check one key. If the key hint is incorrect (the key does not exist, or the
+   password is incorrect) then restic will check all keys, as usual.
+
+   https://github.com/restic/restic/issues/2097
+
+ * Enhancement #2017: Mount: Enforce FUSE Unix permissions with allow-other
+
+   The fuse mount (`restic mount`) now lets the kernel check the permissions of the files within
+   snapshots (this is done through the `DefaultPermissions` FUSE option) when the option
+   `--allow-other` is specified.
+
+   To restore the old behavior, we've added the `--no-default-permissions` option. This allows
+   all users that have access to the mount point to access all files within the snapshots.
+
+   https://github.com/restic/restic/pull/2017
+
+ * Enhancement #2070: Make all commands display timestamps in local time
+
+   Restic used to drop the timezone information from displayed timestamps, it now converts
+   timestamps to local time before printing them so the times can be easily compared to.
+
+   https://github.com/restic/restic/pull/2070
+
+ * Enhancement #2085: Allow --files-from to be specified multiple times
+
+   Before, restic took only the last file specified with `--files-from` into account, this is now
+   corrected.
+
+   https://github.com/restic/restic/issues/2085
+   https://github.com/restic/restic/pull/2086
+
+ * Enhancement #2094: Run command to get password
+
+   We've added the `--password-command` option which allows specifying a command that restic
+   runs every time the password for the repository is needed, so it can be integrated with a
+   password manager or keyring. The option can also be set via the environment variable
+   `$RESTIC_PASSWORD_COMMAND`.
+
+   https://github.com/restic/restic/pull/2094
+
+
 Changelog for restic 0.9.3 (2018-10-13)
 =======================================

@@ -8,7 +136,7 @@ Summary
 -------

 * Fix #1935: Remove truncated files from cache
- * Fix #1978: Do not return an error when the scanner is faster than backup
+ * Fix #1978: Do not return an error when the scanner is slower than backup
 * Enh #1766: Restore: suppress lchown errors when not running as root
 * Enh #1909: Reject files/dirs by name first
 * Enh #1940: Add directory filter to ls command
@@ -20,6 +148,7 @@ Summary
 * Enh #1920: Vendor dependencies with Go 1.11 Modules
 * Enh #1949: Add new command `self-update`
 * Enh #1953: Ls: Add JSON output support for restic ls cmd
+ * Enh #1962: Stream JSON output for ls command

 Details
 -------
@@ -32,7 +161,7 @@ Details

   https://github.com/restic/restic/issues/1935

- * Bugfix #1978: Do not return an error when the scanner is faster than backup
+ * Bugfix #1978: Do not return an error when the scanner is slower than backup

   When restic makes a backup, there's a background task called "scanner" which collects
   information on how many files and directories are to be saved, in order to display progress
@@ -155,6 +284,18 @@ Details

   https://github.com/restic/restic/pull/1953

+ * Enhancement #1962: Stream JSON output for ls command
+
+   The `ls` command now supports JSON output with the global `--json` flag, and this change
+   streams out JSON messages one object at a time rather than en entire array buffered in memory
+   before encoding. The advantage is it allows large listings to be handled efficiently.
+
+   Two message types are printed: snapshots and nodes. A snapshot object will precede node
+   objects which belong to that snapshot. The `struct_type` field can be used to determine which
+   kind of message an object is.
+
+   https://github.com/restic/restic/pull/1962
+

 Changelog for restic 0.9.2 (2018-08-06)
 =======================================
--- a/README.rst
+++ b/README.rst
@@ -3,7 +3,7 @@
 Introduction
 ------------

-restic is a backup program that is fast, efficient and secure.
+restic is a backup program that is fast, efficient and secure. It supports the three major operating systems (Linux, macOS, Windows) and a few smaller ones (FreeBSD, OpenBSD).

 For detailed usage and installation instructions check out the `documentation <https://restic.readthedocs.io/en/latest>`__.

--- a/2
+++ b/2
@@ -1 +1 @@
-0.9.3
+0.9.4
--- a/changelog/0.9.3_2018-10-13/issue-1978
+++ b/changelog/0.9.3_2018-10-13/issue-1978
@@ -1,4 +1,4 @@
-Bugfix: Do not return an error when the scanner is faster than backup
+Bugfix: Do not return an error when the scanner is slower than backup

 When restic makes a backup, there's a background task called "scanner" which
 collects information on how many files and directories are to be saved, in
--- a/changelog/0.9.3_2018-10-13/pull-1962
+++ b/changelog/0.9.3_2018-10-13/pull-1962
@@ -0,0 +1,13 @@
+Enhancement: Stream JSON output for ls command
+
+The `ls` command now supports JSON output with the global `--json`
+flag, and this change streams out JSON messages one object at a time
+rather than en entire array buffered in memory before encoding. The
+advantage is it allows large listings to be handled efficiently.
+
+Two message types are printed: snapshots and nodes. A snapshot
+object will precede node objects which belong to that snapshot.
+The `struct_type` field can be used to determine which kind of
+message an object is.
+
+https://github.com/restic/restic/pull/1962
--- a/changelog/0.9.4_2019-01-06/issue-1605
+++ b/changelog/0.9.4_2019-01-06/issue-1605
@@ -0,0 +1,11 @@
+Enhancement: Concurrent restore
+
+This change significantly improves restore performance, especially
+when using high-latency remote repositories like B2.
+
+The implementation now uses several concurrent threads to download and process
+multiple remote files concurrently. To further reduce restore time, each remote
+file is downloaded using a single repository request.
+
+https://github.com/restic/restic/issues/1605
+https://github.com/restic/restic/pull/1719
--- a/changelog/0.9.4_2019-01-06/issue-1989
+++ b/changelog/0.9.4_2019-01-06/issue-1989
@@ -0,0 +1,7 @@
+Bugfix: Google Cloud Storage: Respect bandwidth limit
+
+The GCS backend did not respect the bandwidth limit configured, a previous
+commit accidentally removed support for it.
+
+https://github.com/restic/restic/issues/1989
+https://github.com/restic/restic/pull/2100
--- a/changelog/0.9.4_2019-01-06/issue-2040
+++ b/changelog/0.9.4_2019-01-06/issue-2040
@@ -0,0 +1,11 @@
+Bugfix: Add host name filter shorthand flag for `stats` command
+
+The default value for `--host` flag was set to 'H' (the shorthand version of
+the flag), this caused the lookup for the latest snapshot to fail.
+
+Add shorthand flag `-H` for `--host` (with empty default so if these flags
+are not specified the latest snapshot will not filter by host name).
+
+Also add shorthand `-H` for `backup` command.
+
+https://github.com/restic/restic/issues/2040
--- a/changelog/0.9.4_2019-01-06/issue-2089
+++ b/changelog/0.9.4_2019-01-06/issue-2089
@@ -0,0 +1,9 @@
+Enhancement: increase granularity of the "keep within" retention policy
+
+The `keep-within` option of the `forget` command now accepts time ranges with
+an hourly granularity. For example, running `restic forget --keep-within 3d12h`
+will keep all the snapshots made within three days and twelve hours from the
+time of the latest snapshot.
+
+https://github.com/restic/restic/issues/2089
+https://github.com/restic/restic/pull/2090
--- a/changelog/0.9.4_2019-01-06/issue-2097
+++ b/changelog/0.9.4_2019-01-06/issue-2097
@@ -0,0 +1,12 @@
+Enhancement: Add key hinting
+
+Added a new option `--key-hint` and corresponding environment variable
+`RESTIC_KEY_HINT`.  The key hint is a key ID to try decrypting first, before
+other keys in the repository.
+
+This change will benefit repositories with many keys; if the correct key hint
+is supplied then restic only needs to check one key.  If the key hint is
+incorrect (the key does not exist, or the password is incorrect) then restic
+will check all keys, as usual.
+
+https://github.com/restic/restic/issues/2097
--- a/changelog/0.9.4_2019-01-06/pull-2017
+++ b/changelog/0.9.4_2019-01-06/pull-2017
@@ -0,0 +1,11 @@
+Enhancement: mount: Enforce FUSE Unix permissions with allow-other
+
+The fuse mount (`restic mount`) now lets the kernel check the permissions of
+the files within snapshots (this is done through the `DefaultPermissions` FUSE
+option) when the option `--allow-other` is specified.
+
+To restore the old behavior, we've added the `--no-default-permissions` option.
+This allows all users that have access to the mount point to access all
+files within the snapshots.
+
+https://github.com/restic/restic/pull/2017
--- a/changelog/0.9.4_2019-01-06/pull-2068
+++ b/changelog/0.9.4_2019-01-06/pull-2068
@@ -0,0 +1,6 @@
+Bugfix: Correctly return error loading data
+
+In one case during `prune` and `check`, an error loading data from the backend is not returned properly. This is now corrected.
+
+https://github.com/restic/restic/pull/2068
+https://github.com/restic/restic/issues/1999#issuecomment-433737921
--- a/changelog/0.9.4_2019-01-06/pull-2070
+++ b/changelog/0.9.4_2019-01-06/pull-2070
@@ -0,0 +1,7 @@
+Enhancement: Make all commands display timestamps in local time
+
+Restic used to drop the timezone information from displayed timestamps, it now
+converts timestamps to local time before printing them so the times can be
+easily compared to.
+
+https://github.com/restic/restic/pull/2070
--- a/changelog/0.9.4_2019-01-06/pull-2086
+++ b/changelog/0.9.4_2019-01-06/pull-2086
@@ -0,0 +1,7 @@
+Enhancement: Allow --files-from to be specified multiple times
+
+Before, restic took only the last file specified with `--files-from` into
+account, this is now corrected.
+
+https://github.com/restic/restic/issues/2085
+https://github.com/restic/restic/pull/2086
--- a/changelog/0.9.4_2019-01-06/pull-2094
+++ b/changelog/0.9.4_2019-01-06/pull-2094
@@ -0,0 +1,8 @@
+Enhancement: Run command to get password
+
+We've added the `--password-command` option which allows specifying a command
+that restic runs every time the password for the repository is needed, so it
+can be integrated with a password manager or keyring. The option can also be
+set via the environment variable `$RESTIC_PASSWORD_COMMAND`.
+
+https://github.com/restic/restic/pull/2094
--- a/changelog/0.9.4_2019-01-06/pull-2095
+++ b/changelog/0.9.4_2019-01-06/pull-2095
@@ -0,0 +1,7 @@
+Bugfix: consistently use local time for snapshots times
+
+By default snapshots created with restic backup were set to local time,
+but when the --time flag was used the provided timestamp was parsed as
+UTC. With this change all snapshots times are set to local time.
+
+https://github.com/restic/restic/pull/2095
--- a/cmd/restic/cmd_backup.go
+++ b/cmd/restic/cmd_backup.go
@@ -45,8 +45,12 @@ given as the arguments.
 	},
 	DisableAutoGenTag: true,
 	RunE: func(cmd *cobra.Command, args []string) error {
-		if backupOptions.Stdin && backupOptions.FilesFrom == "-" {
-			return errors.Fatal("cannot use both `--stdin` and `--files-from -`")
+		if backupOptions.Stdin {
+			for _, filename := range backupOptions.FilesFrom {
+				if filename == "-" {
+					return errors.Fatal("cannot use both `--stdin` and `--files-from -`")
+				}
+			}
 		}

 		var t tomb.Tomb
@@ -75,7 +79,7 @@ type BackupOptions struct {
 	StdinFilename    string
 	Tags             []string
 	Host             string
-	FilesFrom        string
+	FilesFrom        []string
 	TimeStamp        string
 	WithAtime        bool
 }
@@ -97,11 +101,11 @@ func init() {
 	f.StringVar(&backupOptions.StdinFilename, "stdin-filename", "stdin", "file name to use when reading from stdin")
 	f.StringArrayVar(&backupOptions.Tags, "tag", nil, "add a `tag` for the new snapshot (can be specified multiple times)")

-	f.StringVar(&backupOptions.Host, "host", "H", "set the `hostname` for the snapshot manually. To prevent an expensive rescan use the \"parent\" flag")
+	f.StringVarP(&backupOptions.Host, "host", "H", "", "set the `hostname` for the snapshot manually. To prevent an expensive rescan use the \"parent\" flag")
 	f.StringVar(&backupOptions.Host, "hostname", "", "set the `hostname` for the snapshot manually")
 	f.MarkDeprecated("hostname", "use --host")

-	f.StringVar(&backupOptions.FilesFrom, "files-from", "", "read the files to backup from file (can be combined with file args)")
+	f.StringArrayVar(&backupOptions.FilesFrom, "files-from", nil, "read the files to backup from file (can be combined with file args/can be specified multiple times)")
 	f.StringVar(&backupOptions.TimeStamp, "time", "", "time of the backup (ex. '2012-11-01 22:08:41') (default: now)")
 	f.BoolVar(&backupOptions.WithAtime, "with-atime", false, "store the atime for all files and directories")
 }
@@ -175,12 +179,16 @@ func readLinesFromFile(filename string) ([]string, error) {

 // Check returns an error when an invalid combination of options was set.
 func (opts BackupOptions) Check(gopts GlobalOptions, args []string) error {
-	if opts.FilesFrom == "-" && gopts.password == "" {
-		return errors.Fatal("unable to read password from stdin when data is to be read from stdin, use --password-file or $RESTIC_PASSWORD")
+	if gopts.password == "" {
+		for _, filename := range opts.FilesFrom {
+			if filename == "-" {
+				return errors.Fatal("unable to read password from stdin when data is to be read from stdin, use --password-file or $RESTIC_PASSWORD")
+			}
+		}
 	}

 	if opts.Stdin {
-		if opts.FilesFrom != "" {
+		if len(opts.FilesFrom) > 0 {
 			return errors.Fatal("--stdin and --files-from cannot be used together")
 		}

@@ -302,20 +310,25 @@ func collectTargets(opts BackupOptions, args []string) (targets []string, err er
 		return nil, nil
 	}

-	fromfile, err := readLinesFromFile(opts.FilesFrom)
-	if err != nil {
-		return nil, err
-	}
-
-	// expand wildcards
 	var lines []string
-	for _, line := range fromfile {
-		var expanded []string
-		expanded, err := filepath.Glob(line)
+	for _, file := range opts.FilesFrom {
+		fromfile, err := readLinesFromFile(file)
 		if err != nil {
-			return nil, errors.WithMessage(err, fmt.Sprintf("pattern: %s", line))
+			return nil, err
+		}
+
+		// expand wildcards
+		for _, line := range fromfile {
+			var expanded []string
+			expanded, err := filepath.Glob(line)
+			if err != nil {
+				return nil, errors.WithMessage(err, fmt.Sprintf("pattern: %s", line))
+			}
+			if len(expanded) == 0 {
+				Warnf("pattern %q does not match any files, skipping\n", line)
+			}
+			lines = append(lines, expanded...)
 		}
-		lines = append(lines, expanded...)
 	}

 	// merge files from files-from into normal args so we can reuse the normal
@@ -374,7 +387,7 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, term *termstatus.Termina

 	timeStamp := time.Now()
 	if opts.TimeStamp != "" {
-		timeStamp, err = time.Parse(TimeFormat, opts.TimeStamp)
+		timeStamp, err = time.ParseInLocation(TimeFormat, opts.TimeStamp, time.Local)
 		if err != nil {
 			return errors.Fatalf("error in time option: %v\n", err)
 		}
@@ -382,6 +395,12 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, term *termstatus.Termina

 	var t tomb.Tomb

+	term.Print("open repository\n")
+	repo, err := OpenRepository(gopts)
+	if err != nil {
+		return err
+	}
+
 	p := ui.NewBackup(term, gopts.verbosity)

 	// use the terminal for stdout/stderr
@@ -403,12 +422,6 @@ func runBackup(opts BackupOptions, gopts GlobalOptions, term *termstatus.Termina

 	t.Go(func() error { return p.Run(t.Context(gopts.ctx)) })

-	p.V("open repository")
-	repo, err := OpenRepository(gopts)
-	if err != nil {
-		return err
-	}
-
 	p.V("lock repository")
 	lock, err := lockRepo(repo)
 	defer unlockRepo(lock)
--- a/cmd/restic/cmd_dump.go
+++ b/cmd/restic/cmd_dump.go
@@ -4,6 +4,7 @@ import (
 	"context"
 	"fmt"
 	"os"
+	"path"
 	"path/filepath"

 	"github.com/restic/restic/internal/debug"
@@ -47,12 +48,12 @@ func init() {
 	flags.StringArrayVar(&dumpOptions.Paths, "path", nil, "only consider snapshots which include this (absolute) `path` for snapshot ID \"latest\"")
 }

-func splitPath(path string) []string {
-	d, f := filepath.Split(path)
+func splitPath(p string) []string {
+	d, f := path.Split(p)
 	if d == "" || d == "/" {
 		return []string{f}
 	}
-	s := splitPath(filepath.Clean(d))
+	s := splitPath(path.Clean(d))
 	return append(s, f)
 }

--- a/cmd/restic/cmd_find.go
+++ b/cmd/restic/cmd_find.go
@@ -213,7 +213,7 @@ func (s *statefulOutput) PrintObjectNormal(kind, id, nodepath, treeID string, sn
 	} else {
 		Printf(" ... path %s\n", nodepath)
 	}
-	Printf(" ... in snapshot %s (%s)\n", sn.ID().Str(), sn.Time.Format(TimeFormat))
+	Printf(" ... in snapshot %s (%s)\n", sn.ID().Str(), sn.Time.Local().Format(TimeFormat))
 }

 func (s *statefulOutput) PrintObject(kind, id, nodepath, treeID string, sn *restic.Snapshot) {
--- a/cmd/restic/cmd_forget.go
+++ b/cmd/restic/cmd_forget.go
@@ -59,7 +59,7 @@ func init() {
 	f.IntVarP(&forgetOptions.Weekly, "keep-weekly", "w", 0, "keep the last `n` weekly snapshots")
 	f.IntVarP(&forgetOptions.Monthly, "keep-monthly", "m", 0, "keep the last `n` monthly snapshots")
 	f.IntVarP(&forgetOptions.Yearly, "keep-yearly", "y", 0, "keep the last `n` yearly snapshots")
-	f.VarP(&forgetOptions.Within, "keep-within", "", "keep snapshots that are older than `duration` (eg. 1y5m7d) relative to the latest snapshot")
+	f.VarP(&forgetOptions.Within, "keep-within", "", "keep snapshots that are newer than `duration` (eg. 1y5m7d2h) relative to the latest snapshot")

 	f.Var(&forgetOptions.KeepTags, "keep-tag", "keep snapshots with this `taglist` (can be specified multiple times)")
 	f.StringVar(&forgetOptions.Host, "host", "", "only consider snapshots with the given `host`")
--- a/cmd/restic/cmd_key.go
+++ b/cmd/restic/cmd_key.go
@@ -59,7 +59,7 @@ func listKeys(ctx context.Context, s *repository.Repository, gopts GlobalOptions
 			ID:       id.Str(),
 			UserName: k.Username,
 			HostName: k.Hostname,
-			Created:  k.Created.Format(TimeFormat),
+			Created:  k.Created.Local().Format(TimeFormat),
 		}

 		keys = append(keys, key)
--- a/cmd/restic/cmd_ls.go
+++ b/cmd/restic/cmd_ls.go
@@ -64,10 +64,8 @@ func init() {

 type lsSnapshot struct {
 	*restic.Snapshot
-
 	ID         *restic.ID `json:"id"`
 	ShortID    string     `json:"short_id"`
-	Nodes      []lsNode   `json:"nodes"`
 	StructType string     `json:"struct_type"` // "snapshot"
 }

@@ -150,24 +148,22 @@ func runLs(opts LsOptions, gopts GlobalOptions, args []string) error {
 	var (
 		printSnapshot func(sn *restic.Snapshot)
 		printNode     func(path string, node *restic.Node)
-		printFinish   func() error
 	)

 	if gopts.JSON {
-		var lssnapshots []lsSnapshot
+		enc := json.NewEncoder(gopts.stdout)

 		printSnapshot = func(sn *restic.Snapshot) {
-			lss := lsSnapshot{
+			enc.Encode(lsSnapshot{
 				Snapshot:   sn,
 				ID:         sn.ID(),
 				ShortID:    sn.ID().Str(),
 				StructType: "snapshot",
-			}
-			lssnapshots = append(lssnapshots, lss)
+			})
 		}

 		printNode = func(path string, node *restic.Node) {
-			lsn := lsNode{
+			enc.Encode(lsNode{
 				Name:       node.Name,
 				Type:       node.Type,
 				Path:       path,
@@ -179,25 +175,15 @@ func runLs(opts LsOptions, gopts GlobalOptions, args []string) error {
 				AccessTime: node.AccessTime,
 				ChangeTime: node.ChangeTime,
 				StructType: "node",
-			}
-			s := &lssnapshots[len(lssnapshots)-1]
-			s.Nodes = append(s.Nodes, lsn)
-		}
-
-		printFinish = func() error {
-			return json.NewEncoder(gopts.stdout).Encode(lssnapshots)
+			})
 		}
 	} else {
-		// default output methods
 		printSnapshot = func(sn *restic.Snapshot) {
 			Verbosef("snapshot %s of %v filtered by %v at %s):\n", sn.ID().Str(), sn.Paths, dirs, sn.Time)
 		}
 		printNode = func(path string, node *restic.Node) {
 			Printf("%s\n", formatNode(path, node, lsOptions.ListLong))
 		}
-		printFinish = func() error {
-			return nil
-		}
 	}

 	for sn := range FindFilteredSnapshots(ctx, repo, opts.Host, opts.Tags, opts.Paths, args[:1]) {
@@ -240,5 +226,6 @@ func runLs(opts LsOptions, gopts GlobalOptions, args []string) error {
 			return err
 		}
 	}
-	return printFinish()
+
+	return nil
 }
--- a/cmd/restic/cmd_mount.go
+++ b/cmd/restic/cmd_mount.go
@@ -53,13 +53,14 @@ For details please see the documentation for time.Format() at:

 // MountOptions collects all options for the mount command.
 type MountOptions struct {
-	OwnerRoot        bool
-	AllowRoot        bool
-	AllowOther       bool
-	Host             string
-	Tags             restic.TagLists
-	Paths            []string
-	SnapshotTemplate string
+	OwnerRoot            bool
+	AllowRoot            bool
+	AllowOther           bool
+	NoDefaultPermissions bool
+	Host                 string
+	Tags                 restic.TagLists
+	Paths                []string
+	SnapshotTemplate     string
 }

 var mountOptions MountOptions
@@ -71,6 +72,7 @@ func init() {
 	mountFlags.BoolVar(&mountOptions.OwnerRoot, "owner-root", false, "use 'root' as the owner of files and dirs")
 	mountFlags.BoolVar(&mountOptions.AllowRoot, "allow-root", false, "allow root user to access the data in the mounted directory")
 	mountFlags.BoolVar(&mountOptions.AllowOther, "allow-other", false, "allow other users to access the data in the mounted directory")
+	mountFlags.BoolVar(&mountOptions.NoDefaultPermissions, "no-default-permissions", false, "for 'allow-other', ignore Unix permissions and allow users to read all snapshot files")

 	mountFlags.StringVarP(&mountOptions.Host, "host", "H", "", `only consider snapshots for this host`)
 	mountFlags.Var(&mountOptions.Tags, "tag", "only consider snapshots which include this `taglist`")
@@ -118,6 +120,11 @@ func mount(opts MountOptions, gopts GlobalOptions, mountpoint string) error {

 	if opts.AllowOther {
 		mountOptions = append(mountOptions, systemFuse.AllowOther())
+
+		// let the kernel check permissions unless it is explicitly disabled
+		if !opts.NoDefaultPermissions {
+			mountOptions = append(mountOptions, systemFuse.DefaultPermissions())
+		}
 	}

 	c, err := systemFuse.Mount(mountpoint, mountOptions...)
--- a/cmd/restic/cmd_recover.go
+++ b/cmd/restic/cmd_recover.go
@@ -0,0 +1,148 @@
+package main
+
+import (
+	"os"
+	"time"
+
+	"github.com/restic/restic/internal/errors"
+	"github.com/restic/restic/internal/restic"
+	"github.com/spf13/cobra"
+)
+
+var cmdRecover = &cobra.Command{
+	Use:   "recover [flags]",
+	Short: "Recover data from the repository",
+	Long: `
+The "recover" command build a new snapshot from all directories it can find in
+the raw data of the repository. It can be used if, for example, a snapshot has
+been removed by accident with "forget".
+`,
+	DisableAutoGenTag: true,
+	RunE: func(cmd *cobra.Command, args []string) error {
+		return runRecover(globalOptions)
+	},
+}
+
+func init() {
+	cmdRoot.AddCommand(cmdRecover)
+}
+
+func runRecover(gopts GlobalOptions) error {
+	hostname, err := os.Hostname()
+	if err != nil {
+		return err
+	}
+
+	repo, err := OpenRepository(gopts)
+	if err != nil {
+		return err
+	}
+
+	lock, err := lockRepo(repo)
+	defer unlockRepo(lock)
+	if err != nil {
+		return err
+	}
+
+	Verbosef("load index files\n")
+	if err = repo.LoadIndex(gopts.ctx); err != nil {
+		return err
+	}
+
+	// trees maps a tree ID to whether or not it is referenced by a different
+	// tree. If it is not referenced, we have a root tree.
+	trees := make(map[restic.ID]bool)
+
+	for blob := range repo.Index().Each(gopts.ctx) {
+		if blob.Blob.Type != restic.TreeBlob {
+			continue
+		}
+		trees[blob.Blob.ID] = false
+	}
+
+	cur := 0
+	max := len(trees)
+	Verbosef("load %d trees\n\n", len(trees))
+
+	for id := range trees {
+		cur++
+		Verbosef("\rtree (%v/%v)", cur, max)
+
+		if !trees[id] {
+			trees[id] = false
+		}
+
+		tree, err := repo.LoadTree(gopts.ctx, id)
+		if err != nil {
+			Warnf("unable to load tree %v: %v\n", id.Str(), err)
+			continue
+		}
+
+		for _, node := range tree.Nodes {
+			if node.Type != "dir" || node.Subtree == nil {
+				continue
+			}
+
+			subtree := *node.Subtree
+			trees[subtree] = true
+		}
+	}
+	Verbosef("\ndone\n")
+
+	roots := restic.NewIDSet()
+	for id, seen := range trees {
+		if seen {
+			continue
+		}
+
+		roots.Insert(id)
+	}
+
+	Verbosef("found %d roots\n", len(roots))
+
+	tree := restic.NewTree()
+	for id := range roots {
+		var subtreeID = id
+		node := restic.Node{
+			Type:       "dir",
+			Name:       id.Str(),
+			Mode:       0755,
+			Subtree:    &subtreeID,
+			AccessTime: time.Now(),
+			ModTime:    time.Now(),
+			ChangeTime: time.Now(),
+		}
+		tree.Insert(&node)
+	}
+
+	treeID, err := repo.SaveTree(gopts.ctx, tree)
+	if err != nil {
+		return errors.Fatalf("unable to save new tree to the repo: %v", err)
+	}
+
+	err = repo.Flush(gopts.ctx)
+	if err != nil {
+		return errors.Fatalf("unable to save blobs to the repo: %v", err)
+	}
+
+	err = repo.SaveIndex(gopts.ctx)
+	if err != nil {
+		return errors.Fatalf("unable to save new index to the repo: %v", err)
+	}
+
+	sn, err := restic.NewSnapshot([]string{"/recover"}, []string{}, hostname, time.Now())
+	if err != nil {
+		return errors.Fatalf("unable to save snapshot: %v", err)
+	}
+
+	sn.Tree = &treeID
+
+	id, err := repo.SaveJSONUnpacked(gopts.ctx, restic.SnapshotFile, sn)
+	if err != nil {
+		return errors.Fatalf("unable to save snapshot: %v", err)
+	}
+
+	Printf("saved new snapshot %v\n", id.Str())
+
+	return nil
+}
--- a/cmd/restic/cmd_restore.go
+++ b/cmd/restic/cmd_restore.go
@@ -113,8 +113,8 @@ func runRestore(opts RestoreOptions, gopts GlobalOptions, args []string) error {
 	}

 	totalErrors := 0
-	res.Error = func(dir string, node *restic.Node, err error) error {
-		Warnf("ignoring error for %s: %s\n", dir, err)
+	res.Error = func(location string, err error) error {
+		Warnf("ignoring error for %s: %s\n", location, err)
 		totalErrors++
 		return nil
 	}
--- a/cmd/restic/cmd_self_update.go
+++ b/cmd/restic/cmd_self_update.go
@@ -1,4 +1,4 @@
-// +build selfupdate
+// xbuild selfupdate

 package main

@@ -14,7 +14,7 @@ var cmdSelfUpdate = &cobra.Command{
 	Use:   "self-update [flags]",
 	Short: "Update the restic binary",
 	Long: `
-The command "update-restic" downloads the latest stable release of restic from
+The command "self-update" downloads the latest stable release of restic from
 GitHub and replaces the currently running binary. After download, the
 authenticity of the binary is verified using the GPG signature on the release
 files.
@@ -36,16 +36,38 @@ func init() {
 	cmdRoot.AddCommand(cmdSelfUpdate)

 	flags := cmdSelfUpdate.Flags()
-	flags.StringVar(&selfUpdateOptions.Output, "output", os.Args[0], "Save the downloaded file as `filename`")
+	flags.StringVar(&selfUpdateOptions.Output, "output", "", "Save the downloaded file as `filename` (default: running binary itself)")
 }

 func runSelfUpdate(opts SelfUpdateOptions, gopts GlobalOptions, args []string) error {
-	v, err := selfupdate.DownloadLatestStableRelease(gopts.ctx, opts.Output, Verbosef)
+	if opts.Output == "" {
+		file, err := os.Executable()
+		if err != nil {
+			return errors.Wrap(err, "unable to find executable")
+		}
+
+		opts.Output = file
+	}
+
+	fi, err := os.Lstat(opts.Output)
+	if err != nil {
+		return err
+	}
+
+	if !fi.Mode().IsRegular() {
+		return errors.Errorf("output file %v is not a normal file, use --output to specify a different file", opts.Output)
+	}
+
+	Printf("writing restic to %v\n", opts.Output)
+
+	v, err := selfupdate.DownloadLatestStableRelease(gopts.ctx, opts.Output, version, Verbosef)
 	if err != nil {
 		return errors.Fatalf("unable to update restic: %v", err)
 	}

-	Printf("successfully updated restic to version %v\n", v)
+	if v != version {
+		Printf("successfully updated restic to version %v\n", v)
+	}

 	return nil
 }
--- a/cmd/restic/cmd_snapshots.go
+++ b/cmd/restic/cmd_snapshots.go
@@ -184,7 +184,7 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
 	for _, sn := range list {
 		data := snapshot{
 			ID:        sn.ID().Str(),
-			Timestamp: sn.Time.Format(TimeFormat),
+			Timestamp: sn.Time.Local().Format(TimeFormat),
 			Hostname:  sn.Hostname,
 			Tags:      sn.Tags,
 			Paths:     sn.Paths,
@@ -195,7 +195,7 @@ func PrintSnapshots(stdout io.Writer, list restic.Snapshots, reasons []restic.Ke
 			data.Reasons = keepReasons[*id].Matches
 		}

-		if len(sn.Paths) > 1 {
+		if len(sn.Paths) > 1 && !compact {
 			multiline = true
 		}

--- a/cmd/restic/cmd_stats.go
+++ b/cmd/restic/cmd_stats.go
@@ -48,7 +48,7 @@ func init() {
 	cmdRoot.AddCommand(cmdStats)
 	f := cmdStats.Flags()
 	f.StringVar(&countMode, "mode", countModeRestoreSize, "counting mode: restore-size (default), files-by-contents, blobs-per-file, or raw-data")
-	f.StringVar(&snapshotByHost, "host", "H", "filter latest snapshot by this hostname")
+	f.StringVarP(&snapshotByHost, "host", "H", "", "filter latest snapshot by this hostname")
 }

 func runStats(gopts GlobalOptions, args []string) error {
@@ -296,7 +296,7 @@ type statsContainer struct {
 	// blobs that have been seen as a part of the file
 	fileBlobs map[string]restic.IDSet

-	// blobs and blobsSeen are used to count indiviudal
+	// blobs and blobsSeen are used to count individual
 	// unique blobs, independent of references to files
 	blobs, blobsSeen restic.BlobSet
 }
--- a/cmd/restic/format.go
+++ b/cmd/restic/format.go
@@ -21,7 +21,7 @@ func formatBytes(c uint64) string {
 	case c > 1<<10:
 		return fmt.Sprintf("%.3f KiB", b/(1<<10))
 	default:
-		return fmt.Sprintf("%dB", c)
+		return fmt.Sprintf("%d B", c)
 	}
 }

@@ -90,6 +90,6 @@ func formatNode(path string, n *restic.Node, long bool) string {

 	return fmt.Sprintf("%s %5d %5d %6d %s %s%s",
 		mode|n.Mode, n.UID, n.GID, n.Size,
-		n.ModTime.Format(TimeFormat), path,
+		n.ModTime.Local().Format(TimeFormat), path,
 		target)
 }
--- a/cmd/restic/global.go
+++ b/cmd/restic/global.go
@@ -34,26 +34,29 @@ import (
 	"github.com/restic/restic/internal/errors"

 	"golang.org/x/crypto/ssh/terminal"
+	"os/exec"
 )

-var version = "0.9.3"
+var version = "0.9.4"

 // TimeFormat is the format used for all timestamps printed by restic.
 const TimeFormat = "2006-01-02 15:04:05"

 // GlobalOptions hold all global options for restic.
 type GlobalOptions struct {
-	Repo          string
-	PasswordFile  string
-	Quiet         bool
-	Verbose       int
-	NoLock        bool
-	JSON          bool
-	CacheDir      string
-	NoCache       bool
-	CACerts       []string
-	TLSClientCert string
-	CleanupCache  bool
+	Repo            string
+	PasswordFile    string
+	PasswordCommand string
+	KeyHint         string
+	Quiet           bool
+	Verbose         int
+	NoLock          bool
+	JSON            bool
+	CacheDir        string
+	NoCache         bool
+	CACerts         []string
+	TLSClientCert   string
+	CleanupCache    bool

 	LimitUploadKb   int
 	LimitDownloadKb int
@@ -91,6 +94,8 @@ func init() {
 	f := cmdRoot.PersistentFlags()
 	f.StringVarP(&globalOptions.Repo, "repo", "r", os.Getenv("RESTIC_REPOSITORY"), "repository to backup to or restore from (default: $RESTIC_REPOSITORY)")
 	f.StringVarP(&globalOptions.PasswordFile, "password-file", "p", os.Getenv("RESTIC_PASSWORD_FILE"), "read the repository password from a file (default: $RESTIC_PASSWORD_FILE)")
+	f.StringVarP(&globalOptions.KeyHint, "key-hint", "", os.Getenv("RESTIC_KEY_HINT"), "key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)")
+	f.StringVarP(&globalOptions.PasswordCommand, "password-command", "", os.Getenv("RESTIC_PASSWORD_COMMAND"), "specify a shell command to obtain a password (default: $RESTIC_PASSWORD_COMMAND)")
 	f.BoolVarP(&globalOptions.Quiet, "quiet", "q", false, "do not output comprehensive progress report")
 	f.CountVarP(&globalOptions.Verbose, "verbose", "v", "be verbose (specify --verbose multiple times or level `n`)")
 	f.BoolVar(&globalOptions.NoLock, "no-lock", false, "do not lock the repo, this allows some operations on read-only repos")
@@ -179,7 +184,6 @@ func Printf(format string, args ...interface{}) {
 	_, err := fmt.Fprintf(globalOptions.stdout, format, args...)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "unable to write to stdout: %v\n", err)
-		Exit(100)
 	}
 }

@@ -220,7 +224,6 @@ func Warnf(format string, args ...interface{}) {
 	_, err := fmt.Fprintf(globalOptions.stderr, format, args...)
 	if err != nil {
 		fmt.Fprintf(os.Stderr, "unable to write to stderr: %v\n", err)
-		Exit(100)
 	}
 }

@@ -236,7 +239,23 @@ func Exitf(exitcode int, format string, args ...interface{}) {
 }

 // resolvePassword determines the password to be used for opening the repository.
-func resolvePassword(opts GlobalOptions, env string) (string, error) {
+func resolvePassword(opts GlobalOptions) (string, error) {
+	if opts.PasswordFile != "" && opts.PasswordCommand != "" {
+		return "", errors.Fatalf("Password file and command are mutually exclusive options")
+	}
+	if opts.PasswordCommand != "" {
+		args, err := backend.SplitShellStrings(opts.PasswordCommand)
+		if err != nil {
+			return "", err
+		}
+		cmd := exec.Command(args[0], args[1:]...)
+		cmd.Stderr = os.Stderr
+		output, err := cmd.Output()
+		if err != nil {
+			return "", err
+		}
+		return (strings.TrimSpace(string(output))), nil
+	}
 	if opts.PasswordFile != "" {
 		s, err := textfile.Read(opts.PasswordFile)
 		if os.IsNotExist(errors.Cause(err)) {
@@ -245,7 +264,7 @@ func resolvePassword(opts GlobalOptions, env string) (string, error) {
 		return strings.TrimSpace(string(s)), errors.Wrap(err, "Readfile")
 	}

-	if pwd := os.Getenv(env); pwd != "" {
+	if pwd := os.Getenv("RESTIC_PASSWORD"); pwd != "" {
 		return pwd, nil
 	}

@@ -353,7 +372,7 @@ func OpenRepository(opts GlobalOptions) (*repository.Repository, error) {
 		return nil, err
 	}

-	err = s.SearchKey(opts.ctx, opts.password, maxKeys)
+	err = s.SearchKey(opts.ctx, opts.password, maxKeys, opts.KeyHint)
 	if err != nil {
 		return nil, err
 	}
@@ -363,7 +382,9 @@ func OpenRepository(opts GlobalOptions) (*repository.Repository, error) {
 		if len(id) > 8 {
 			id = id[:8]
 		}
-		Verbosef("repository %v opened successfully, password is correct\n", id)
+		if !opts.JSON {
+			Verbosef("repository %v opened successfully, password is correct\n", id)
+		}
 	}

 	if opts.NoCache {
--- a/cmd/restic/main.go
+++ b/cmd/restic/main.go
@@ -54,7 +54,7 @@ directories in an encrypted repository stored on different backends.
 		if c.Name() == "version" {
 			return nil
 		}
-		pwd, err := resolvePassword(globalOptions, "RESTIC_PASSWORD")
+		pwd, err := resolvePassword(globalOptions)
 		if err != nil {
 			fmt.Fprintf(os.Stderr, "Resolving password failed: %v\n", err)
 			Exit(1)
--- a/doc/020_installation.rst
+++ b/doc/020_installation.rst
@@ -30,12 +30,12 @@ command.
 Arch Linux
 ==========

-On `Arch Linux <https://www.archlinux.org/>`__, there is a package called ``restic-git``
-which can be installed from AUR, e.g. with ``pacaur``:
+On `Arch Linux <https://www.archlinux.org/>`__, there is a package called ``restic``
+installed from the official community repos, e.g. with ``pacman -S``:

 .. code-block:: console

-    $ pacaur -S restic-git
+    $ pacman -S restic

 Debian
 ======
@@ -72,7 +72,7 @@ macOS
 =====

 If you are using macOS, you can install restic using the
-`homebrew <http://brew.sh/>`__ package manager:
+`homebrew <https://brew.sh/>`__ package manager:

 .. code-block:: console

--- a/doc/030_preparing_a_new_repo.rst
+++ b/doc/030_preparing_a_new_repo.rst
@@ -21,6 +21,19 @@ using a local repository; the remaining sections of this chapter cover all the
 other options. You can skip to the next chapter once you've read the relevant
 section here.

+For automated backups, restic accepts the repository location in the
+environment variable ``RESTIC_REPOSITORY``. For the password, several options
+exist:
+
+ * Setting the environment variable ``RESTIC_PASSWORD``
+
+ * Specifying the path to a file with the password via the option
+   ``--password-file`` or the environment variable ``RESTIC_PASSWORD_FILE``
+
+ * Configuring a program to be called when the password is needed via the
+   option ``--password-command`` or the environment variable
+   ``RESTIC_PASSWORD_COMMAND``
+
 Local
 *****

@@ -41,11 +54,6 @@ command and enter the same password twice:
   Remembering your password is important! If you lose it, you won't be
   able to access data stored in the repository.

-For automated backups, restic accepts the repository location in the
-environment variable ``RESTIC_REPOSITORY``. The password can be read
-from a file (via the option ``--password-file`` or the environment variable
-``RESTIC_PASSWORD_FILE``) or the environment variable ``RESTIC_PASSWORD``.
-
 SFTP
 ****

@@ -298,10 +306,10 @@ dashboard in on the "Buckets" page when signed into your B2 account:

 .. code-block:: console

-    $ export B2_ACCOUNT_ID=<MY_ACCOUNT_ID>
+    $ export B2_ACCOUNT_ID=<MY_APPLICATION_KEY_ID>
    $ export B2_ACCOUNT_KEY=<MY_SECRET_ACCOUNT_KEY>

-.. note:: In case you want to use Backblaze Application Keys replace <MY_ACCOUNT_ID> and <MY_SECRET_ACCOUNT_KEY> with <applicationKeyId> and <applicationKey> respectively.
+.. note:: In case you want to use Backblaze Application Keys replace <MY_APPLICATION_KEY_ID> and <MY_SECRET_ACCOUNT_KEY> with <applicationKeyId> and <applicationKey> respectively.

 You can then initialize a repository stored at Backblaze B2. If the
 bucket does not exist yet and the credentials you passed to restic have the
--- a/doc/040_backup.rst
+++ b/doc/040_backup.rst
@@ -204,9 +204,12 @@ backup ``/sys`` or ``/dev`` on a Linux system:

    $ restic -r /srv/restic-repo backup --one-file-system /

+.. note:: ``--one-file-system`` is currently unsupported on Windows, and will
+    cause the backup to immediately fail with an error.
+
 By using the ``--files-from`` option you can read the files you want to
-backup from a file. This is especially useful if a lot of files have to
-be backed up that are not in the same folder or are maybe pre-filtered
+backup from one or more files. This is especially useful if a lot of files have
+to be backed up that are not in the same folder or are maybe pre-filtered
 by other software.

 For example maybe you want to backup files which have a name that matches a
--- a/doc/045_working_with_repos.rst
+++ b/doc/045_working_with_repos.rst
@@ -66,7 +66,7 @@ backup with the intention to make you restore malicious data:

 .. code-block:: console

-    $ sudo echo "boom" >> backup/index/d795ffa99a8ab8f8e42cec1f814df4e48b8f49129360fb57613df93739faee97
+    $ echo "boom" >> backup/index/d795ffa99a8ab8f8e42cec1f814df4e48b8f49129360fb57613df93739faee97

 In order to detect these things, it is a good idea to regularly use the
 ``check`` command to test whether everything is alright, your precious
--- a/doc/060_forget.rst
+++ b/doc/060_forget.rst
@@ -168,8 +168,9 @@ The ``forget`` command accepts the following parameters:
   this option (can be specified multiple times).
 -  ``--keep-within duration`` keep all snapshots which have been made within
   the duration of the latest snapshot. ``duration`` needs to be a number of
-   years, months, and days, e.g. ``2y5m7d`` will keep all snapshots made in the
-   two years, five months, and seven days before the latest snapshot.
+   years, months, days, and hours, e.g. ``2y5m7d3h`` will keep all snapshots
+   made in the two years, five months, seven days, and three hours before the
+   latest snapshot.

 Multiple policies will be ORed together so as to be as inclusive as possible
 for keeping snapshots.
--- a/doc/090_participating.rst
+++ b/doc/090_participating.rst
@@ -85,7 +85,7 @@ back end is contained in `Design <https://restic.readthedocs.io/en/latest/design
 If you'd like to start contributing to restic, but don't know exactly
 what do to, have a look at this great article by Dave Cheney:
 `Suggestions for contributing to an Open Source
-project <http://dave.cheney.net/2016/03/12/suggestions-for-contributing-to-an-open-source-project>`__
+project <https://dave.cheney.net/2016/03/12/suggestions-for-contributing-to-an-open-source-project>`__
 A few issues have been tagged with the label ``help wanted``, you can
 start looking at those:
 https://github.com/restic/restic/labels/help%20wanted
@@ -113,7 +113,7 @@ Compatibility

 Backward compatibility for backups is important so that our users are
 always able to restore saved data. Therefore restic follows `Semantic
-Versioning <http://semver.org>`__ to clearly define which versions are
+Versioning <https://semver.org>`__ to clearly define which versions are
 compatible. The repository and data structures contained therein are
 considered the "Public API" in the sense of Semantic Versioning. This
 goes for all released versions of restic, this may not be the case for
@@ -128,7 +128,7 @@ prior versions.
 Building documentation
 **********************

-The restic documentation is built with `Sphinx <http://sphinx-doc.org>`__,
+The restic documentation is built with `Sphinx <https://www.sphinx-doc.org>`__,
 therefore building it locally requires a recent Python version and requirements listed in ``doc/requirements.txt``.
 This example will guide you through the process using `virtualenv <https://virtualenv.pypa.io>`__:

--- a/doc/bash-completion.sh
+++ b/doc/bash-completion.sh
@@ -275,6 +275,7 @@ _restic_backup()
    flags+=("-h")
    local_nonpersistent_flags+=("--help")
    flags+=("--host=")
+    two_word_flags+=("-H")
    local_nonpersistent_flags+=("--host=")
    flags+=("--one-file-system")
    flags+=("-x")
@@ -295,12 +296,14 @@ _restic_backup()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -343,12 +346,14 @@ _restic_cache()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -385,12 +390,14 @@ _restic_cat()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -435,12 +442,14 @@ _restic_check()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -479,12 +488,14 @@ _restic_diff()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -528,12 +539,14 @@ _restic_dump()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -600,12 +613,14 @@ _restic_find()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -681,12 +696,14 @@ _restic_forget()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -729,12 +746,14 @@ _restic_generate()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -771,12 +790,14 @@ _restic_init()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -815,12 +836,14 @@ _restic_key()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -857,12 +880,14 @@ _restic_list()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -911,12 +936,14 @@ _restic_ls()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -956,12 +983,14 @@ _restic_migrate()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1001,6 +1030,8 @@ _restic_mount()
    flags+=("--host=")
    two_word_flags+=("-H")
    local_nonpersistent_flags+=("--host=")
+    flags+=("--no-default-permissions")
+    local_nonpersistent_flags+=("--no-default-permissions")
    flags+=("--owner-root")
    local_nonpersistent_flags+=("--owner-root")
    flags+=("--path=")
@@ -1013,12 +1044,14 @@ _restic_mount()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1055,12 +1088,14 @@ _restic_prune()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1097,12 +1132,58 @@ _restic_rebuild-index()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
+    flags+=("--password-file=")
+    two_word_flags+=("-p")
+    flags+=("--quiet")
+    flags+=("-q")
+    flags+=("--repo=")
+    two_word_flags+=("-r")
+    flags+=("--tls-client-cert=")
+    flags+=("--verbose")
+    flags+=("-v")
+
+    must_have_one_flag=()
+    must_have_one_noun=()
+    noun_aliases=()
+}
+
+_restic_recover()
+{
+    last_command="restic_recover"
+
+    command_aliases=()
+
+    commands=()
+
+    flags=()
+    two_word_flags=()
+    local_nonpersistent_flags=()
+    flags_with_completion=()
+    flags_completion=()
+
+    flags+=("--help")
+    flags+=("-h")
+    local_nonpersistent_flags+=("--help")
+    flags+=("--cacert=")
+    flags+=("--cache-dir=")
+    flags+=("--cleanup-cache")
+    flags+=("--json")
+    flags+=("--key-hint=")
+    flags+=("--limit-download=")
+    flags+=("--limit-upload=")
+    flags+=("--no-cache")
+    flags+=("--no-lock")
+    flags+=("--option=")
+    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1157,12 +1238,14 @@ _restic_restore()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1201,12 +1284,14 @@ _restic_self-update()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1255,12 +1340,14 @@ _restic_snapshots()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1294,6 +1381,7 @@ _restic_stats()
    flags+=("-h")
    local_nonpersistent_flags+=("--help")
    flags+=("--host=")
+    two_word_flags+=("-H")
    local_nonpersistent_flags+=("--host=")
    flags+=("--mode=")
    local_nonpersistent_flags+=("--mode=")
@@ -1301,12 +1389,14 @@ _restic_stats()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1356,12 +1446,14 @@ _restic_tag()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1400,12 +1492,14 @@ _restic_unlock()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1442,12 +1536,14 @@ _restic_version()
    flags+=("--cache-dir=")
    flags+=("--cleanup-cache")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
@@ -1487,6 +1583,7 @@ _restic_root_command()
    commands+=("mount")
    commands+=("prune")
    commands+=("rebuild-index")
+    commands+=("recover")
    commands+=("restore")
    commands+=("self-update")
    commands+=("snapshots")
@@ -1508,12 +1605,14 @@ _restic_root_command()
    flags+=("-h")
    local_nonpersistent_flags+=("--help")
    flags+=("--json")
+    flags+=("--key-hint=")
    flags+=("--limit-download=")
    flags+=("--limit-upload=")
    flags+=("--no-cache")
    flags+=("--no-lock")
    flags+=("--option=")
    two_word_flags+=("-o")
+    flags+=("--password-command=")
    flags+=("--password-file=")
    two_word_flags+=("-p")
    flags+=("--quiet")
--- a/doc/cache.rst
+++ b/doc/cache.rst
@@ -9,7 +9,7 @@ Versions
 ========

 The cache directory is selected according to the `XDG base dir specification
-<http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html>`__.
+<https://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html>`__.
 Each repository has its own cache sub-directory, consisting of the repository ID
 which is chosen at ``init``. All cache directories for different repos are
 independent of each other.
--- a/doc/design.rst
+++ b/doc/design.rst
@@ -276,7 +276,7 @@ the IV for counter mode and the nonce for Poly1305. This operation needs
 three keys: A 32 byte for AES-256 for encryption, a 16 byte AES key and
 a 16 byte key for Poly1305. For details see the original paper `The
 Poly1305-AES message-authentication
-code <http://cr.yp.to/mac/poly1305-20050329.pdf>`__ by Dan Bernstein.
+code <https://cr.yp.to/mac/poly1305-20050329.pdf>`__ by Dan Bernstein.
 The data is then encrypted with AES-256 and afterwards a message
 authentication code (MAC) is computed over the ciphertext, everything is
 then stored as IV \|\| CIPHERTEXT \|\| MAC.
--- a/doc/faq.rst
+++ b/doc/faq.rst
@@ -27,7 +27,7 @@ strictly necessary. With high probability this is duplicate data. In
 order to clean it up, the command ``restic prune`` can be used. The
 cause of this bug is not yet known.

-I ran a ``restic`` command but it is not working as intented, what do I do now?
+I ran a ``restic`` command but it is not working as intended, what do I do now?
 -------------------------------------------------------------------------------

 If you are running a restic command and it is not working as you hoped it would,
@@ -45,7 +45,7 @@ $ restic backup --exclude "~/documents" ~
 This command will result in a complete backup of the current logged in user's home directory and it won't exclude the folder ``~/documents/`` - which is not what the user wanted to achieve.
 The problem is how the path to ``~/documents`` is passed to restic.

-In order to spot an issue like this, you can make use of the following ruby command preceeding your restic command.
+In order to spot an issue like this, you can make use of the following ruby command preceding your restic command.

 ::

@@ -71,7 +71,7 @@ Restic handles globbing and expansion in the following ways:
 -  Globbing is only expanded for lines read via ``--files-from``
 -  Environment variables are not expanded in the file read via ``--files-from``
 -  ``*`` is expanded for paths read via ``--files-from``
-  E.g. For backup targets given to restic as arguments on the shell, neither glob expansion nor shell variable replacement is done. If restic is called as ``restic backup '*' '$HOME'``, it will try to backup the literal file(s)/dir(s) ``*`` and ``$HOME``
+-  e.g. For backup targets given to restic as arguments on the shell, neither glob expansion nor shell variable replacement is done. If restic is called as ``restic backup '*' '$HOME'``, it will try to backup the literal file(s)/dir(s) ``*`` and ``$HOME``
 -  Double-asterisk ``**`` only works in exclude patterns as this is a custom extension built into restic; the shell must not expand it


@@ -172,4 +172,4 @@ The following may work:
 Why does restic perform so poorly on Windows?
 ---------------------------------------------

-In some cases the realtime protection of antivirus software can interfere with restic's operations. If you are experiencing bad performace you can try to temporarily disable your antivirus software to find out if it is the cause for your performance problems.
+In some cases the real-time protection of antivirus software can interfere with restic's operations. If you are experiencing bad performance you can try to temporarily disable your antivirus software to find out if it is the cause for your performance problems.
--- a/doc/man/restic-backup.1
+++ b/doc/man/restic-backup.1
@@ -37,8 +37,8 @@ given as the arguments.
    takes filename[:header], exclude contents of directories containing filename (except filename itself) if header of that file is as provided (can be specified multiple times)

 .PP
-\fB\-\-files\-from\fP=""
-    read the files to backup from file (can be combined with file args)
+\fB\-\-files\-from\fP=[]
+    read the files to backup from file (can be combined with file args/can be specified multiple times)

 .PP
 \fB\-f\fP, \fB\-\-force\fP[=false]
@@ -49,7 +49,7 @@ given as the arguments.
    help for backup

 .PP
-\fB\-\-host\fP="H"
+\fB\-H\fP, \fB\-\-host\fP=""
    set the \fB\fChostname\fR for the snapshot manually. To prevent an expensive rescan use the "parent" flag

 .PP
@@ -98,6 +98,10 @@ given as the arguments.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -118,6 +122,10 @@ given as the arguments.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-cache.1
+++ b/doc/man/restic-cache.1
@@ -53,6 +53,10 @@ The "cache" command allows listing and cleaning local cache directories.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -73,6 +77,10 @@ The "cache" command allows listing and cleaning local cache directories.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-cat.1
+++ b/doc/man/restic-cat.1
@@ -41,6 +41,10 @@ The "cat" command is used to print internal objects to stdout.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -61,6 +65,10 @@ The "cat" command is used to print internal objects to stdout.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-check.1
+++ b/doc/man/restic-check.1
@@ -62,6 +62,10 @@ repository and not use a local cache.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -82,6 +86,10 @@ repository and not use a local cache.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-diff.1
+++ b/doc/man/restic-diff.1
@@ -61,6 +61,10 @@ T  The type was changed, e.g. a file was made a symlink
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -81,6 +85,10 @@ T  The type was changed, e.g. a file was made a symlink
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-dump.1
+++ b/doc/man/restic-dump.1
@@ -58,6 +58,10 @@ repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -78,6 +82,10 @@ repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-find.1
+++ b/doc/man/restic-find.1
@@ -91,6 +91,10 @@ It can also be used to search for restic blobs or trees for troubleshooting.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -111,6 +115,10 @@ It can also be used to search for restic blobs or trees for troubleshooting.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-forget.1
+++ b/doc/man/restic-forget.1
@@ -48,7 +48,7 @@ data after 'forget' was run successfully, see the 'prune' command.

 .PP
 \fB\-\-keep\-within\fP=
-    keep snapshots that are older than \fB\fCduration\fR (eg. 1y5m7d) relative to the latest snapshot
+    keep snapshots that are newer than \fB\fCduration\fR (eg. 1y5m7d2h) relative to the latest snapshot

 .PP
 \fB\-\-keep\-tag\fP=[]
@@ -104,6 +104,10 @@ data after 'forget' was run successfully, see the 'prune' command.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -124,6 +128,10 @@ data after 'forget' was run successfully, see the 'prune' command.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-generate.1
+++ b/doc/man/restic-generate.1
@@ -54,6 +54,10 @@ and the auto\-completion files for bash and zsh).
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -74,6 +78,10 @@ and the auto\-completion files for bash and zsh).
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-init.1
+++ b/doc/man/restic-init.1
@@ -41,6 +41,10 @@ The "init" command initializes a new repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -61,6 +65,10 @@ The "init" command initializes a new repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-key.1
+++ b/doc/man/restic-key.1
@@ -45,6 +45,10 @@ The "key" command manages keys (passwords) for accessing the repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -65,6 +69,10 @@ The "key" command manages keys (passwords) for accessing the repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-list.1
+++ b/doc/man/restic-list.1
@@ -41,6 +41,10 @@ The "list" command allows listing objects in the repository based on type.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -61,6 +65,10 @@ The "list" command allows listing objects in the repository based on type.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-ls.1
+++ b/doc/man/restic-ls.1
@@ -76,6 +76,10 @@ a path separator); paths use the forward slash '/' as separator.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -96,6 +100,10 @@ a path separator); paths use the forward slash '/' as separator.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-migrate.1
+++ b/doc/man/restic-migrate.1
@@ -46,6 +46,10 @@ name is explicitly given, a list of migrations that can be applied is printed.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -66,6 +70,10 @@ name is explicitly given, a list of migrations that can be applied is printed.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-mount.1
+++ b/doc/man/restic-mount.1
@@ -68,6 +68,10 @@ For details please see the documentation for time.Format() at:
 \fB\-H\fP, \fB\-\-host\fP=""
    only consider snapshots for this host

+.PP
+\fB\-\-no\-default\-permissions\fP[=false]
+    for 'allow\-other', ignore Unix permissions and allow users to read all snapshot files
+
 .PP
 \fB\-\-owner\-root\fP[=false]
    use 'root' as the owner of files and dirs
@@ -102,6 +106,10 @@ For details please see the documentation for time.Format() at:
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -122,6 +130,10 @@ For details please see the documentation for time.Format() at:
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-prune.1
+++ b/doc/man/restic-prune.1
@@ -42,6 +42,10 @@ referenced and therefore not needed any more.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -62,6 +66,10 @@ referenced and therefore not needed any more.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-rebuild-index.1
+++ b/doc/man/restic-rebuild-index.1
@@ -42,6 +42,10 @@ repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -62,6 +66,10 @@ repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-recover.1
+++ b/doc/man/restic-recover.1
@@ -0,0 +1,97 @@
+.TH "restic backup" "1" "Jan 2017" "generated by `restic generate`" "" 
+.nh
+.ad l
+
+
+.SH NAME
+.PP
+restic\-recover \- Recover data from the repository
+
+
+.SH SYNOPSIS
+.PP
+\fBrestic recover [flags]\fP
+
+
+.SH DESCRIPTION
+.PP
+The "recover" command build a new snapshot from all directories it can find in
+the raw data of the repository. It can be used if, for example, a snapshot has
+been removed by accident with "forget".
+
+
+.SH OPTIONS
+.PP
+\fB\-h\fP, \fB\-\-help\fP[=false]
+    help for recover
+
+
+.SH OPTIONS INHERITED FROM PARENT COMMANDS
+.PP
+\fB\-\-cacert\fP=[]
+    \fB\fCfile\fR to load root certificates from (default: use system certificates)
+
+.PP
+\fB\-\-cache\-dir\fP=""
+    set the cache directory. (default: use system default cache directory)
+
+.PP
+\fB\-\-cleanup\-cache\fP[=false]
+    auto remove old cache directories
+
+.PP
+\fB\-\-json\fP[=false]
+    set output mode to JSON for commands that support it
+
+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
+.PP
+\fB\-\-limit\-download\fP=0
+    limits downloads to a maximum rate in KiB/s. (default: unlimited)
+
+.PP
+\fB\-\-limit\-upload\fP=0
+    limits uploads to a maximum rate in KiB/s. (default: unlimited)
+
+.PP
+\fB\-\-no\-cache\fP[=false]
+    do not use a local cache
+
+.PP
+\fB\-\-no\-lock\fP[=false]
+    do not lock the repo, this allows some operations on read\-only repos
+
+.PP
+\fB\-o\fP, \fB\-\-option\fP=[]
+    set extended option (\fB\fCkey=value\fR, can be specified multiple times)
+
+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
+.PP
+\fB\-p\fP, \fB\-\-password\-file\fP=""
+    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
+
+.PP
+\fB\-q\fP, \fB\-\-quiet\fP[=false]
+    do not output comprehensive progress report
+
+.PP
+\fB\-r\fP, \fB\-\-repo\fP=""
+    repository to backup to or restore from (default: $RESTIC\_REPOSITORY)
+
+.PP
+\fB\-\-tls\-client\-cert\fP=""
+    path to a file containing PEM encoded TLS client certificate and private key
+
+.PP
+\fB\-v\fP, \fB\-\-verbose\fP[=0]
+    be verbose (specify \-\-verbose multiple times or level \fB\fCn\fR)
+
+
+.SH SEE ALSO
+.PP
+\fBrestic(1)\fP
--- a/doc/man/restic-restore.1
+++ b/doc/man/restic-restore.1
@@ -74,6 +74,10 @@ repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -94,6 +98,10 @@ repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-self-update.1
+++ b/doc/man/restic-self-update.1
@@ -15,7 +15,7 @@ restic\-self\-update \- Update the restic binary

 .SH DESCRIPTION
 .PP
-The command "update\-restic" downloads the latest stable release of restic from
+The command "self\-update" downloads the latest stable release of restic from
 GitHub and replaces the currently running binary. After download, the
 authenticity of the binary is verified using the GPG signature on the release
 files.
@@ -27,8 +27,8 @@ files.
    help for self\-update

 .PP
-\fB\-\-output\fP="./restic\-generate.temp"
-    Save the downloaded file as \fB\fCfilename\fR
+\fB\-\-output\fP=""
+    Save the downloaded file as \fB\fCfilename\fR (default: running binary itself)


 .SH OPTIONS INHERITED FROM PARENT COMMANDS
@@ -48,6 +48,10 @@ files.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -68,6 +72,10 @@ files.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-snapshots.1
+++ b/doc/man/restic-snapshots.1
@@ -61,6 +61,10 @@ The "snapshots" command lists all snapshots stored in the repository.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -81,6 +85,10 @@ The "snapshots" command lists all snapshots stored in the repository.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-stats.1
+++ b/doc/man/restic-stats.1
@@ -52,7 +52,7 @@ Refer to the online manual for more details about each mode.
    help for stats

 .PP
-\fB\-\-host\fP="H"
+\fB\-H\fP, \fB\-\-host\fP=""
    filter latest snapshot by this hostname

 .PP
@@ -77,6 +77,10 @@ Refer to the online manual for more details about each mode.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -97,6 +101,10 @@ Refer to the online manual for more details about each mode.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-tag.1
+++ b/doc/man/restic-tag.1
@@ -72,6 +72,10 @@ When no snapshot\-ID is given, all snapshots matching the host, tag and path fil
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -92,6 +96,10 @@ When no snapshot\-ID is given, all snapshots matching the host, tag and path fil
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-unlock.1
+++ b/doc/man/restic-unlock.1
@@ -45,6 +45,10 @@ The "unlock" command removes stale locks that have been created by other restic
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -65,6 +69,10 @@ The "unlock" command removes stale locks that have been created by other restic
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic-version.1
+++ b/doc/man/restic-version.1
@@ -42,6 +42,10 @@ and the version of this software.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -62,6 +66,10 @@ and the version of this software.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
--- a/doc/man/restic.1
+++ b/doc/man/restic.1
@@ -40,6 +40,10 @@ directories in an encrypted repository stored on different backends.
 \fB\-\-json\fP[=false]
    set output mode to JSON for commands that support it

+.PP
+\fB\-\-key\-hint\fP=""
+    key ID of key to try decrypting first (default: $RESTIC\_KEY\_HINT)
+
 .PP
 \fB\-\-limit\-download\fP=0
    limits downloads to a maximum rate in KiB/s. (default: unlimited)
@@ -60,6 +64,10 @@ directories in an encrypted repository stored on different backends.
 \fB\-o\fP, \fB\-\-option\fP=[]
    set extended option (\fB\fCkey=value\fR, can be specified multiple times)

+.PP
+\fB\-\-password\-command\fP=""
+    specify a shell command to obtain a password (default: $RESTIC\_PASSWORD\_COMMAND)
+
 .PP
 \fB\-p\fP, \fB\-\-password\-file\fP=""
    read the repository password from a file (default: $RESTIC\_PASSWORD\_FILE)
@@ -83,4 +91,4 @@ directories in an encrypted repository stored on different backends.

 .SH SEE ALSO
 .PP
-\fBrestic\-backup(1)\fP, \fBrestic\-cache(1)\fP, \fBrestic\-cat(1)\fP, \fBrestic\-check(1)\fP, \fBrestic\-diff(1)\fP, \fBrestic\-dump(1)\fP, \fBrestic\-find(1)\fP, \fBrestic\-forget(1)\fP, \fBrestic\-generate(1)\fP, \fBrestic\-init(1)\fP, \fBrestic\-key(1)\fP, \fBrestic\-list(1)\fP, \fBrestic\-ls(1)\fP, \fBrestic\-migrate(1)\fP, \fBrestic\-mount(1)\fP, \fBrestic\-prune(1)\fP, \fBrestic\-rebuild\-index(1)\fP, \fBrestic\-restore(1)\fP, \fBrestic\-self\-update(1)\fP, \fBrestic\-snapshots(1)\fP, \fBrestic\-stats(1)\fP, \fBrestic\-tag(1)\fP, \fBrestic\-unlock(1)\fP, \fBrestic\-version(1)\fP
+\fBrestic\-backup(1)\fP, \fBrestic\-cache(1)\fP, \fBrestic\-cat(1)\fP, \fBrestic\-check(1)\fP, \fBrestic\-diff(1)\fP, \fBrestic\-dump(1)\fP, \fBrestic\-find(1)\fP, \fBrestic\-forget(1)\fP, \fBrestic\-generate(1)\fP, \fBrestic\-init(1)\fP, \fBrestic\-key(1)\fP, \fBrestic\-list(1)\fP, \fBrestic\-ls(1)\fP, \fBrestic\-migrate(1)\fP, \fBrestic\-mount(1)\fP, \fBrestic\-prune(1)\fP, \fBrestic\-rebuild\-index(1)\fP, \fBrestic\-recover(1)\fP, \fBrestic\-restore(1)\fP, \fBrestic\-self\-update(1)\fP, \fBrestic\-snapshots(1)\fP, \fBrestic\-stats(1)\fP, \fBrestic\-tag(1)\fP, \fBrestic\-unlock(1)\fP, \fBrestic\-version(1)\fP
--- a/doc/manual_rest.rst
+++ b/doc/manual_rest.rst
@@ -47,6 +47,7 @@ Usage help is available:
          --cleanup-cache            auto remove old cache directories
      -h, --help                     help for restic
          --json                     set output mode to JSON for commands that support it
+          --key-hint string          key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
          --limit-download int       limits downloads to a maximum rate in KiB/s. (default: unlimited)
          --limit-upload int         limits uploads to a maximum rate in KiB/s. (default: unlimited)
          --no-cache                 do not use a local cache
@@ -80,7 +81,7 @@ command:
          --exclude-caches                   excludes cache directories that are marked with a CACHEDIR.TAG file
          --exclude-file file                read exclude patterns from a file (can be specified multiple times)
          --exclude-if-present stringArray   takes filename[:header], exclude contents of directories containing filename (except filename itself) if header of that file is as provided (can be specified multiple times)
-          --files-from string                read the files to backup from file (can be combined with file args)
+          --files-from string                read the files to backup from file (can be combined with file args/can be specified multiple times)
      -f, --force                            force re-reading the target files/directories (overrides the "parent" flag)
      -h, --help                             help for backup
          --hostname hostname                set the hostname for the snapshot manually. To prevent an expensive rescan use the "parent" flag
@@ -97,6 +98,7 @@ command:
          --cache-dir string         set the cache directory. (default: use system default cache directory)
          --cleanup-cache            auto remove old cache directories
          --json                     set output mode to JSON for commands that support it
+          --key-hint string          key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
          --limit-download int       limits downloads to a maximum rate in KiB/s. (default: unlimited)
          --limit-upload int         limits uploads to a maximum rate in KiB/s. (default: unlimited)
          --no-cache                 do not use a local cache
@@ -179,7 +181,7 @@ locks with the following command:
    d369ccc7d126594950bf74f0a348d5d98d9e99f3215082eb69bf02dc9b3e464c

 The ``find`` command searches for a given
-`pattern <http://golang.org/pkg/path/filepath/#Match>`__ in the
+`pattern <https://golang.org/pkg/path/filepath/#Match>`__ in the
 repository.

 .. code-block:: console
--- a/doc/zsh-completion.zsh
+++ b/doc/zsh-completion.zsh
@@ -7,7 +7,7 @@ case $state in
  level1)
    case $words[1] in
      restic)
-        _arguments '1: :(backup cache cat check diff dump find forget generate help init key list ls migrate mount options prune rebuild-index restore self-update snapshots stats tag unlock version)'
+        _arguments '1: :(backup cache cat check diff dump find forget generate help init key list ls migrate mount options prune rebuild-index recover restore self-update snapshots stats tag unlock version)'
      ;;
      *)
        _arguments '*: :_files'
--- a/docker/Dockerfile
+++ b/docker/Dockerfile
@@ -2,6 +2,6 @@ FROM alpine:latest

 COPY restic /usr/bin

-RUN apk add --update --no-cache ca-certificates fuse
+RUN apk add --update --no-cache ca-certificates fuse openssh-client

 ENTRYPOINT ["/usr/bin/restic"]
--- a/go.mod
+++ b/go.mod
@@ -15,6 +15,7 @@ require (
 	github.com/golang/protobuf v1.2.0 // indirect
 	github.com/google/go-cmp v0.2.0
 	github.com/gopherjs/gopherjs v0.0.0-20180825215210-0210a2f0f73c // indirect
+	github.com/hashicorp/golang-lru v0.5.0
 	github.com/inconshreveable/mousetrap v1.0.0 // indirect
 	github.com/jtolds/gls v4.2.1+incompatible // indirect
 	github.com/juju/ratelimit v1.0.1
--- a/go.sum
+++ b/go.sum
@@ -26,6 +26,8 @@ github.com/google/go-cmp v0.2.0 h1:+dTQ8DZQJz0Mb/HjFlkptS1FeQ4cWSnN941F8aEG4SQ=
 github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
 github.com/gopherjs/gopherjs v0.0.0-20180825215210-0210a2f0f73c h1:16eHWuMGvCjSfgRJKqIzapE78onvvTbdi1rMkU00lZw=
 github.com/gopherjs/gopherjs v0.0.0-20180825215210-0210a2f0f73c/go.mod h1:wJfORRmW1u3UXTncJ5qlYoELFm8eSnnEO6hX4iZ3EWY=
+github.com/hashicorp/golang-lru v0.5.0 h1:CL2msUPvZTLb5O648aiLNJw3hnBxN2+1Jq8rCOH9wdo=
+github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
 github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
 github.com/jtolds/gls v4.2.1+incompatible h1:fSuqC+Gmlu6l/ZYAoZzx2pyucC8Xza35fpRVWLVmUEE=
--- a/helpers/prepare-release/main.go
+++ b/helpers/prepare-release/main.go
@@ -398,6 +398,7 @@ func updateDocker(outputDir, version string) {
 	cmd := fmt.Sprintf("bzcat %s/restic_%s_linux_amd64.bz2 > restic", outputDir, version)
 	run("sh", "-c", cmd)
 	run("chmod", "+x", "restic")
+	run("docker", "pull", "alpine:latest")
 	run("docker", "build", "--rm", "--tag", "restic/restic:latest", "-f", "docker/Dockerfile", ".")
 	run("docker", "tag", "restic/restic:latest", "restic/restic:"+version)
 }
--- a/internal/backend/gs/gs.go
+++ b/internal/backend/gs/gs.go
@@ -15,6 +15,7 @@ import (
 	"github.com/restic/restic/internal/debug"
 	"github.com/restic/restic/internal/restic"

+	"golang.org/x/oauth2"
 	"golang.org/x/oauth2/google"
 	"google.golang.org/api/googleapi"
 	storage "google.golang.org/api/storage/v1"
@@ -40,8 +41,17 @@ type Backend struct {
 // Ensure that *Backend implements restic.Backend.
 var _ restic.Backend = &Backend{}

-func getStorageService() (*storage.Service, error) {
-	client, err := google.DefaultClient(context.TODO(), storage.DevstorageReadWriteScope)
+func getStorageService(rt http.RoundTripper) (*storage.Service, error) {
+	// create a new HTTP client
+	httpClient := &http.Client{
+		Transport: rt,
+	}
+
+	// create a now context with the HTTP client stored at the oauth2.HTTPClient key
+	ctx := context.WithValue(context.Background(), oauth2.HTTPClient, httpClient)
+
+	// use this context
+	client, err := google.DefaultClient(ctx, storage.DevstorageReadWriteScope)
 	if err != nil {
 		return nil, err
 	}
@@ -59,7 +69,7 @@ const defaultListMaxItems = 1000
 func open(cfg Config, rt http.RoundTripper) (*Backend, error) {
 	debug.Log("open, config %#v", cfg)

-	service, err := getStorageService()
+	service, err := getStorageService(rt)
 	if err != nil {
 		return nil, errors.Wrap(err, "getStorageService")
 	}
--- a/internal/backend/rclone/backend.go
+++ b/internal/backend/rclone/backend.go
@@ -210,7 +210,7 @@ func New(cfg Config, lim limiter.Limiter) (*Backend, error) {

 	// send an HTTP request to the base URL, see if the server is there
 	client := &http.Client{
-		Transport: tr,
+		Transport: debug.RoundTripper(tr),
 		Timeout:   60 * time.Second,
 	}

@@ -255,7 +255,7 @@ func Open(cfg Config, lim limiter.Limiter) (*Backend, error) {
 		URL:         url,
 	}

-	restBackend, err := rest.Open(restConfig, be.tr)
+	restBackend, err := rest.Open(restConfig, debug.RoundTripper(be.tr))
 	if err != nil {
 		return nil, err
 	}
@@ -283,7 +283,7 @@ func Create(cfg Config) (*Backend, error) {
 		URL:         url,
 	}

-	restBackend, err := rest.Create(restConfig, be.tr)
+	restBackend, err := rest.Create(restConfig, debug.RoundTripper(be.tr))
 	if err != nil {
 		_ = be.Close()
 		return nil, err
--- a/internal/cache/dir.go
+++ b/internal/cache/dir.go
@@ -32,7 +32,7 @@ func xdgCacheDir() (string, error) {
 func windowsCacheDir() (string, error) {
 	appdata := os.Getenv("LOCALAPPDATA")
 	if appdata == "" {
-		return "", errors.New("unable to locate cache directory (APPDATA unset)")
+		return "", errors.New("unable to locate cache directory (LOCALAPPDATA unset)")
 	}
 	return filepath.Join(appdata, "restic"), nil
 }
--- a/internal/cache/file.go
+++ b/internal/cache/file.go
@@ -127,7 +127,8 @@ func (c *Cache) Save(h restic.Handle, rd io.Reader) error {
 	if n <= crypto.Extension {
 		_ = f.Close()
 		_ = c.Remove(h)
-		return errors.Errorf("trying to cache truncated file %v", h)
+		debug.Log("trying to cache truncated file %v, removing", h)
+		return nil
 	}

 	if err = f.Close(); err != nil {
--- a/internal/checker/checker.go
+++ b/internal/checker/checker.go
@@ -7,13 +7,12 @@ import (
 	"os"
 	"sync"

-	"github.com/restic/restic/internal/errors"
-	"github.com/restic/restic/internal/restic"
-	"golang.org/x/sync/errgroup"
-
 	"github.com/restic/restic/internal/debug"
+	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/pack"
 	"github.com/restic/restic/internal/repository"
+	"github.com/restic/restic/internal/restic"
+	"golang.org/x/sync/errgroup"
 )

 // Checker runs various checks on a repository. It is advisable to create an
@@ -652,7 +651,7 @@ func checkPack(ctx context.Context, r restic.Repository, id restic.ID) error {
 	debug.Log("checking pack %v", id)
 	h := restic.Handle{Type: restic.DataFile, Name: id.String()}

-	packfile, hash, size, err := repository.DownloadAndHash(ctx, r, h)
+	packfile, hash, size, err := repository.DownloadAndHash(ctx, r.Backend(), h)
 	if err != nil {
 		return errors.Wrap(err, "checkPack")
 	}
--- a/internal/checker/checker_test.go
+++ b/internal/checker/checker_test.go
@@ -330,7 +330,7 @@ func TestCheckerModifiedData(t *testing.T) {

 	beError := &errorBackend{Backend: repo.Backend()}
 	checkRepo := repository.New(beError)
-	test.OK(t, checkRepo.SearchKey(context.TODO(), test.TestPassword, 5))
+	test.OK(t, checkRepo.SearchKey(context.TODO(), test.TestPassword, 5, ""))

 	chkr := checker.New(checkRepo)

--- a/internal/index/index.go
+++ b/internal/index/index.go
@@ -5,14 +5,13 @@ import (
 	"context"
 	"fmt"
 	"os"
+	"sync"

 	"github.com/restic/restic/internal/debug"
-	"github.com/restic/restic/internal/list"
+	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/pack"
 	"github.com/restic/restic/internal/restic"
-	"github.com/restic/restic/internal/worker"
-
-	"github.com/restic/restic/internal/errors"
+	"golang.org/x/sync/errgroup"
 )

 // Pack contains information about the contents of a pack.
@@ -35,38 +34,118 @@ func newIndex() *Index {
 	}
 }

+const listPackWorkers = 10
+
+// Lister lists files and their contents
+type Lister interface {
+	// List runs fn for all files of type t in the repo.
+	List(ctx context.Context, t restic.FileType, fn func(restic.ID, int64) error) error
+
+	// ListPack returns the list of blobs saved in the pack id and the length
+	// of the file as stored in the backend.
+	ListPack(ctx context.Context, id restic.ID, size int64) ([]restic.Blob, int64, error)
+}
+
 // New creates a new index for repo from scratch. InvalidFiles contains all IDs
 // of files  that cannot be listed successfully.
-func New(ctx context.Context, repo restic.Repository, ignorePacks restic.IDSet, p *restic.Progress) (idx *Index, invalidFiles restic.IDs, err error) {
+func New(ctx context.Context, repo Lister, ignorePacks restic.IDSet, p *restic.Progress) (idx *Index, invalidFiles restic.IDs, err error) {
 	p.Start()
 	defer p.Done()

-	ch := make(chan worker.Job)
-	go list.AllPacks(ctx, repo, ignorePacks, ch)
+	type Job struct {
+		PackID restic.ID
+		Size   int64
+	}
+
+	type Result struct {
+		Error   error
+		PackID  restic.ID
+		Size    int64
+		Entries []restic.Blob
+	}
+
+	inputCh := make(chan Job)
+	outputCh := make(chan Result)
+	wg, ctx := errgroup.WithContext(ctx)
+
+	// list the files in the repo, send to inputCh
+	wg.Go(func() error {
+		defer close(inputCh)
+		return repo.List(ctx, restic.DataFile, func(id restic.ID, size int64) error {
+			if ignorePacks.Has(id) {
+				return nil
+			}
+
+			job := Job{
+				PackID: id,
+				Size:   size,
+			}
+
+			select {
+			case inputCh <- job:
+			case <-ctx.Done():
+			}
+			return nil
+		})
+	})
+
+	// run the workers listing the files, read from inputCh, send to outputCh
+	var workers sync.WaitGroup
+	for i := 0; i < listPackWorkers; i++ {
+		workers.Add(1)
+		go func() {
+			defer workers.Done()
+			for job := range inputCh {
+				res := Result{PackID: job.PackID}
+				res.Entries, res.Size, res.Error = repo.ListPack(ctx, job.PackID, job.Size)
+
+				select {
+				case outputCh <- res:
+				case <-ctx.Done():
+					return
+				}
+			}
+		}()
+	}
+
+	// wait until all the workers are done, then close outputCh
+	wg.Go(func() error {
+		workers.Wait()
+		close(outputCh)
+		return nil
+	})

 	idx = newIndex()

-	for job := range ch {
+	for res := range outputCh {
 		p.Report(restic.Stat{Blobs: 1})
-
-		j := job.Result.(list.Result)
-		if job.Error != nil {
-			cause := errors.Cause(job.Error)
+		if res.Error != nil {
+			cause := errors.Cause(res.Error)
 			if _, ok := cause.(pack.InvalidFileError); ok {
-				invalidFiles = append(invalidFiles, j.PackID())
+				invalidFiles = append(invalidFiles, res.PackID)
 				continue
 			}

-			fmt.Fprintf(os.Stderr, "pack file cannot be listed %v: %v\n", j.PackID(), job.Error)
+			fmt.Fprintf(os.Stderr, "pack file cannot be listed %v: %v\n", res.PackID, res.Error)
 			continue
 		}

-		debug.Log("pack %v contains %d blobs", j.PackID(), len(j.Entries()))
+		debug.Log("pack %v contains %d blobs", res.PackID, len(res.Entries))

-		err := idx.AddPack(j.PackID(), j.Size(), j.Entries())
+		err := idx.AddPack(res.PackID, res.Size, res.Entries)
 		if err != nil {
 			return nil, nil, err
 		}
+
+		select {
+		case <-ctx.Done(): // an error occurred
+		default:
+		}
+	}
+
+	err = wg.Wait()
+	if err != nil {
+		return nil, nil, err
 	}

 	return idx, invalidFiles, nil
@@ -89,7 +168,13 @@ type indexJSON struct {
 	Packs      []packJSON `json:"packs"`
 }

-func loadIndexJSON(ctx context.Context, repo restic.Repository, id restic.ID) (*indexJSON, error) {
+// ListLoader allows listing files and their content, in addition to loading and unmarshaling JSON files.
+type ListLoader interface {
+	Lister
+	LoadJSONUnpacked(context.Context, restic.FileType, restic.ID, interface{}) error
+}
+
+func loadIndexJSON(ctx context.Context, repo ListLoader, id restic.ID) (*indexJSON, error) {
 	debug.Log("process index %v\n", id)

 	var idx indexJSON
@@ -102,7 +187,7 @@ func loadIndexJSON(ctx context.Context, repo restic.Repository, id restic.ID) (*
 }

 // Load creates an index by loading all index files from the repo.
-func Load(ctx context.Context, repo restic.Repository, p *restic.Progress) (*Index, error) {
+func Load(ctx context.Context, repo ListLoader, p *restic.Progress) (*Index, error) {
 	debug.Log("loading indexes")

 	p.Start()
@@ -259,8 +344,13 @@ func (idx *Index) FindBlob(h restic.BlobHandle) (result []Location, err error) {

 const maxEntries = 3000

+// Saver saves structures as JSON.
+type Saver interface {
+	SaveJSONUnpacked(ctx context.Context, t restic.FileType, item interface{}) (restic.ID, error)
+}
+
 // Save writes the complete index to the repo.
-func (idx *Index) Save(ctx context.Context, repo restic.Repository, supersedes restic.IDs) (restic.IDs, error) {
+func (idx *Index) Save(ctx context.Context, repo Saver, supersedes restic.IDs) (restic.IDs, error) {
 	debug.Log("pack files: %d\n", len(idx.Packs))

 	var indexIDs []restic.ID
--- a/internal/index/index_test.go
+++ b/internal/index/index_test.go
@@ -2,10 +2,12 @@ package index

 import (
 	"context"
+	"sync"
 	"testing"
 	"time"

 	"github.com/restic/restic/internal/checker"
+	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/repository"
 	"github.com/restic/restic/internal/restic"
 	"github.com/restic/restic/internal/test"
@@ -48,7 +50,7 @@ func TestIndexNew(t *testing.T) {
 	repo, cleanup := createFilledRepo(t, 3, 0)
 	defer cleanup()

-	idx, _, err := New(context.TODO(), repo, restic.NewIDSet(), nil)
+	idx, invalid, err := New(context.TODO(), repo, restic.NewIDSet(), nil)
 	if err != nil {
 		t.Fatalf("New() returned error %v", err)
 	}
@@ -57,9 +59,102 @@ func TestIndexNew(t *testing.T) {
 		t.Fatalf("New() returned nil index")
 	}

+	if len(invalid) > 0 {
+		t.Fatalf("New() returned invalid files: %v", invalid)
+	}
+
 	validateIndex(t, repo, idx)
 }

+type ErrorRepo struct {
+	restic.Repository
+	MaxListFiles int
+
+	MaxPacks      int
+	MaxPacksMutex sync.Mutex
+}
+
+// List returns an error after repo.MaxListFiles files.
+func (repo *ErrorRepo) List(ctx context.Context, t restic.FileType, fn func(restic.ID, int64) error) error {
+	if repo.MaxListFiles == 0 {
+		return errors.New("test error, max is zero")
+	}
+
+	max := repo.MaxListFiles
+	return repo.Repository.List(ctx, t, func(id restic.ID, size int64) error {
+		if max == 0 {
+			return errors.New("test error, max reached zero")
+		}
+
+		max--
+		return fn(id, size)
+	})
+}
+
+// ListPack returns an error after repo.MaxPacks files.
+func (repo *ErrorRepo) ListPack(ctx context.Context, id restic.ID, size int64) ([]restic.Blob, int64, error) {
+	repo.MaxPacksMutex.Lock()
+	max := repo.MaxPacks
+	if max > 0 {
+		repo.MaxPacks--
+	}
+	repo.MaxPacksMutex.Unlock()
+
+	if max == 0 {
+		return nil, 0, errors.New("test list pack error")
+	}
+
+	return repo.Repository.ListPack(ctx, id, size)
+}
+
+func TestIndexNewListErrors(t *testing.T) {
+	repo, cleanup := createFilledRepo(t, 3, 0)
+	defer cleanup()
+
+	for _, max := range []int{0, 3, 5} {
+		errRepo := &ErrorRepo{
+			Repository:   repo,
+			MaxListFiles: max,
+		}
+		idx, invalid, err := New(context.TODO(), errRepo, restic.NewIDSet(), nil)
+		if err == nil {
+			t.Errorf("expected error not found, got nil")
+		}
+
+		if idx != nil {
+			t.Errorf("expected nil index, got %v", idx)
+		}
+
+		if len(invalid) != 0 {
+			t.Errorf("expected empty invalid list, got %v", invalid)
+		}
+	}
+}
+
+func TestIndexNewPackErrors(t *testing.T) {
+	repo, cleanup := createFilledRepo(t, 3, 0)
+	defer cleanup()
+
+	for _, max := range []int{0, 3, 5} {
+		errRepo := &ErrorRepo{
+			Repository: repo,
+			MaxPacks:   max,
+		}
+		idx, invalid, err := New(context.TODO(), errRepo, restic.NewIDSet(), nil)
+		if err == nil {
+			t.Errorf("expected error not found, got nil")
+		}
+
+		if idx != nil {
+			t.Errorf("expected nil index, got %v", idx)
+		}
+
+		if len(invalid) != 0 {
+			t.Errorf("expected empty invalid list, got %v", invalid)
+		}
+	}
+}
+
 func TestIndexLoad(t *testing.T) {
 	repo, cleanup := createFilledRepo(t, 3, 0)
 	defer cleanup()
@@ -186,7 +281,7 @@ func BenchmarkIndexSave(b *testing.B) {
 }

 func TestIndexDuplicateBlobs(t *testing.T) {
-	repo, cleanup := createFilledRepo(t, 3, 0.01)
+	repo, cleanup := createFilledRepo(t, 3, 0.05)
 	defer cleanup()

 	idx, _, err := New(context.TODO(), repo, restic.NewIDSet(), nil)
@@ -252,6 +347,7 @@ func TestIndexSave(t *testing.T) {
 	}

 	ctx, cancel := context.WithCancel(context.TODO())
+	defer cancel()

 	errCh := make(chan error)
 	go checker.Structure(ctx, errCh)
--- a/internal/list/list.go
+++ b/internal/list/list.go
@@ -1,79 +0,0 @@
-package list
-
-import (
-	"context"
-
-	"github.com/restic/restic/internal/restic"
-	"github.com/restic/restic/internal/worker"
-)
-
-const listPackWorkers = 10
-
-// Lister combines lists packs in a repo and blobs in a pack.
-type Lister interface {
-	List(context.Context, restic.FileType, func(restic.ID, int64) error) error
-	ListPack(context.Context, restic.ID, int64) ([]restic.Blob, int64, error)
-}
-
-// Result is returned in the channel from LoadBlobsFromAllPacks.
-type Result struct {
-	packID  restic.ID
-	size    int64
-	entries []restic.Blob
-}
-
-// PackID returns the pack ID of this result.
-func (l Result) PackID() restic.ID {
-	return l.packID
-}
-
-// Size returns the size of the pack.
-func (l Result) Size() int64 {
-	return l.size
-}
-
-// Entries returns a list of all blobs saved in the pack.
-func (l Result) Entries() []restic.Blob {
-	return l.entries
-}
-
-// AllPacks sends the contents of all packs to ch.
-func AllPacks(ctx context.Context, repo Lister, ignorePacks restic.IDSet, ch chan<- worker.Job) {
-	type fileInfo struct {
-		id   restic.ID
-		size int64
-	}
-
-	f := func(ctx context.Context, job worker.Job) (interface{}, error) {
-		packInfo := job.Data.(fileInfo)
-		entries, size, err := repo.ListPack(ctx, packInfo.id, packInfo.size)
-
-		return Result{
-			packID:  packInfo.id,
-			size:    size,
-			entries: entries,
-		}, err
-	}
-
-	jobCh := make(chan worker.Job)
-	wp := worker.New(ctx, listPackWorkers, f, jobCh, ch)
-
-	go func() {
-		defer close(jobCh)
-
-		_ = repo.List(ctx, restic.DataFile, func(id restic.ID, size int64) error {
-			if ignorePacks.Has(id) {
-				return nil
-			}
-
-			select {
-			case jobCh <- worker.Job{Data: fileInfo{id: id, size: size}, Result: Result{packID: id}}:
-			case <-ctx.Done():
-				return ctx.Err()
-			}
-			return nil
-		})
-	}()
-
-	wp.Wait()
-}
--- a/internal/repository/key.go
+++ b/internal/repository/key.go
@@ -112,9 +112,26 @@ func OpenKey(ctx context.Context, s *Repository, name string, password string) (
 // given password. If none could be found, ErrNoKeyFound is returned. When
 // maxKeys is reached, ErrMaxKeysReached is returned. When setting maxKeys to
 // zero, all keys in the repo are checked.
-func SearchKey(ctx context.Context, s *Repository, password string, maxKeys int) (k *Key, err error) {
+func SearchKey(ctx context.Context, s *Repository, password string, maxKeys int, keyHint string) (k *Key, err error) {
 	checked := 0

+	if len(keyHint) > 0 {
+		id, err := restic.Find(s.Backend(), restic.KeyFile, keyHint)
+
+		if err == nil {
+			key, err := OpenKey(ctx, s, id, password)
+
+			if err == nil {
+				debug.Log("successfully opened hinted key %v", id)
+				return key, nil
+			}
+
+			debug.Log("could not open hinted key %v", id)
+		} else {
+			debug.Log("Could not find hinted key %v", keyHint)
+		}
+	}
+
 	listCtx, cancel := context.WithCancel(ctx)
 	defer cancel()

--- a/internal/repository/packer_manager_test.go
+++ b/internal/repository/packer_manager_test.go
@@ -92,6 +92,9 @@ func fillPacks(t testing.TB, rnd *randReader, be Saver, pm *packerManager, buf [
 		}

 		n, err := packer.Add(restic.DataBlob, id, buf)
+		if err != nil {
+			t.Fatal(err)
+		}
 		if n != l {
 			t.Errorf("Add() returned invalid number of bytes: want %v, got %v", n, l)
 		}
--- a/internal/repository/repack.go
+++ b/internal/repository/repack.go
@@ -6,11 +6,10 @@ import (
 	"os"

 	"github.com/restic/restic/internal/debug"
+	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/fs"
 	"github.com/restic/restic/internal/pack"
 	"github.com/restic/restic/internal/restic"
-
-	"github.com/restic/restic/internal/errors"
 )

 // Repack takes a list of packs together with a list of blobs contained in
@@ -24,7 +23,7 @@ func Repack(ctx context.Context, repo restic.Repository, packs restic.IDSet, kee
 		// load the complete pack into a temp file
 		h := restic.Handle{Type: restic.DataFile, Name: packID.String()}

-		tempfile, hash, packLength, err := DownloadAndHash(ctx, repo, h)
+		tempfile, hash, packLength, err := DownloadAndHash(ctx, repo.Backend(), h)
 		if err != nil {
 			return nil, errors.Wrap(err, "Repack")
 		}
--- a/internal/repository/repository.go
+++ b/internal/repository/repository.go
@@ -9,16 +9,15 @@ import (
 	"io"
 	"os"

+	"github.com/restic/restic/internal/backend"
 	"github.com/restic/restic/internal/cache"
+	"github.com/restic/restic/internal/crypto"
+	"github.com/restic/restic/internal/debug"
 	"github.com/restic/restic/internal/errors"
 	"github.com/restic/restic/internal/fs"
 	"github.com/restic/restic/internal/hashing"
-	"github.com/restic/restic/internal/restic"
-
-	"github.com/restic/restic/internal/backend"
-	"github.com/restic/restic/internal/crypto"
-	"github.com/restic/restic/internal/debug"
 	"github.com/restic/restic/internal/pack"
+	"github.com/restic/restic/internal/restic"
 )

 // Repository is used to access a repository in a backend.
@@ -511,8 +510,8 @@ func LoadIndex(ctx context.Context, repo restic.Repository, id restic.ID) (*Inde

 // SearchKey finds a key with the supplied password, afterwards the config is
 // read and parsed. It tries at most maxKeys key files in the repo.
-func (r *Repository) SearchKey(ctx context.Context, password string, maxKeys int) error {
-	key, err := SearchKey(ctx, r, password, maxKeys)
+func (r *Repository) SearchKey(ctx context.Context, password string, maxKeys int, keyHint string) error {
+	key, err := SearchKey(ctx, r, password, maxKeys, keyHint)
 	if err != nil {
 		return err
 	}
@@ -694,16 +693,21 @@ func (r *Repository) SaveTree(ctx context.Context, t *restic.Tree) (restic.ID, e
 	return id, err
 }

+// Loader allows loading data from a backend.
+type Loader interface {
+	Load(ctx context.Context, h restic.Handle, length int, offset int64, fn func(rd io.Reader) error) error
+}
+
 // DownloadAndHash is all-in-one helper to download content of the file at h to a temporary filesystem location
 // and calculate ID of the contents. Returned (temporary) file is positioned at the beginning of the file;
 // it is reponsibility of the caller to close and delete the file.
-func DownloadAndHash(ctx context.Context, repo restic.Repository, h restic.Handle) (tmpfile *os.File, hash restic.ID, size int64, err error) {
+func DownloadAndHash(ctx context.Context, be Loader, h restic.Handle) (tmpfile *os.File, hash restic.ID, size int64, err error) {
 	tmpfile, err = fs.TempFile("", "restic-temp-")
 	if err != nil {
 		return nil, restic.ID{}, -1, errors.Wrap(err, "TempFile")
 	}

-	err = repo.Backend().Load(ctx, h, 0, 0, func(rd io.Reader) (ierr error) {
+	err = be.Load(ctx, h, 0, 0, func(rd io.Reader) (ierr error) {
 		_, ierr = tmpfile.Seek(0, io.SeekStart)
 		if ierr == nil {
 			ierr = tmpfile.Truncate(0)
@@ -716,6 +720,11 @@ func DownloadAndHash(ctx context.Context, repo restic.Repository, h restic.Handl
 		hash = restic.IDFromHash(hrd.Sum(nil))
 		return ierr
 	})
+	if err != nil {
+		tmpfile.Close()
+		os.Remove(tmpfile.Name())
+		return nil, restic.ID{}, -1, errors.Wrap(err, "Load")
+	}

 	_, err = tmpfile.Seek(0, io.SeekStart)
 	if err != nil {
--- a/internal/repository/repository_test.go
+++ b/internal/repository/repository_test.go
@@ -11,6 +11,8 @@ import (
 	"time"

 	"github.com/restic/restic/internal/archiver"
+	"github.com/restic/restic/internal/errors"
+	"github.com/restic/restic/internal/fs"
 	"github.com/restic/restic/internal/repository"
 	"github.com/restic/restic/internal/restic"
 	rtest "github.com/restic/restic/internal/test"
@@ -392,3 +394,107 @@ func TestRepositoryIncrementalIndex(t *testing.T) {
 		}
 	}
 }
+
+type backend struct {
+	rd io.Reader
+}
+
+func (be backend) Load(ctx context.Context, h restic.Handle, length int, offset int64, fn func(rd io.Reader) error) error {
+	return fn(be.rd)
+}
+
+type retryBackend struct {
+	buf []byte
+}
+
+func (be retryBackend) Load(ctx context.Context, h restic.Handle, length int, offset int64, fn func(rd io.Reader) error) error {
+	err := fn(bytes.NewReader(be.buf[:len(be.buf)/2]))
+	if err != nil {
+		return err
+	}
+
+	return fn(bytes.NewReader(be.buf))
+}
+
+func TestDownloadAndHash(t *testing.T) {
+	buf := make([]byte, 5*1024*1024+881)
+	_, err := io.ReadFull(rnd, buf)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	var tests = []struct {
+		be   repository.Loader
+		want []byte
+	}{
+		{
+			be:   backend{rd: bytes.NewReader(buf)},
+			want: buf,
+		},
+		{
+			be:   retryBackend{buf: buf},
+			want: buf,
+		},
+	}
+
+	for _, test := range tests {
+		t.Run("", func(t *testing.T) {
+			f, id, size, err := repository.DownloadAndHash(context.TODO(), test.be, restic.Handle{})
+			if err != nil {
+				t.Error(err)
+			}
+
+			want := restic.Hash(test.want)
+			if !want.Equal(id) {
+				t.Errorf("wrong hash returned, want %v, got %v", want.Str(), id.Str())
+			}
+
+			if size != int64(len(test.want)) {
+				t.Errorf("wrong size returned, want %v, got %v", test.want, size)
+			}
+
+			err = f.Close()
+			if err != nil {
+				t.Error(err)
+			}
+
+			err = fs.RemoveIfExists(f.Name())
+			if err != nil {
+				t.Fatal(err)
+			}
+		})
+	}
+}
+
+type errorReader struct {
+	err error
+}
+
+func (er errorReader) Read(p []byte) (n int, err error) {
+	return 0, er.err
+}
+
+func TestDownloadAndHashErrors(t *testing.T) {
+	var tests = []struct {
+		be  repository.Loader
+		err string
+	}{
+		{
+			be:  backend{rd: errorReader{errors.New("test error 1")}},
+			err: "test error 1",
+		},
+	}
+
+	for _, test := range tests {
+		t.Run("", func(t *testing.T) {
+			_, _, _, err := repository.DownloadAndHash(context.TODO(), test.be, restic.Handle{})
+			if err == nil {
+				t.Fatalf("wanted error %q, got nil", test.err)
+			}
+
+			if errors.Cause(err).Error() != test.err {
+				t.Fatalf("wanted error %q, got %q", test.err, err)
+			}
+		})
+	}
+}
--- a/internal/repository/testing.go
+++ b/internal/repository/testing.go
@@ -99,7 +99,7 @@ func TestOpenLocal(t testing.TB, dir string) (r restic.Repository) {
 	}

 	repo := New(be)
-	err = repo.SearchKey(context.TODO(), test.TestPassword, 10)
+	err = repo.SearchKey(context.TODO(), test.TestPassword, 10, "")
 	if err != nil {
 		t.Fatal(err)
 	}
--- a/internal/restic/duration.go
+++ b/internal/restic/duration.go
@@ -10,9 +10,9 @@ import (
 )

 // Duration is similar to time.Duration, except it only supports larger ranges
-// like days, months, and years.
+// like hours, days, months, and years.
 type Duration struct {
-	Days, Months, Years int
+	Hours, Days, Months, Years int
 }

 func (d Duration) String() string {
@@ -29,6 +29,10 @@ func (d Duration) String() string {
 		s += fmt.Sprintf("%dd", d.Days)
 	}

+	if d.Hours != 0 {
+		s += fmt.Sprintf("%dh", d.Hours)
+	}
+
 	return s
 }

@@ -73,7 +77,7 @@ func nextNumber(input string) (num int, rest string, err error) {
 }

 // ParseDuration parses a duration from a string. The format is:
-//    6y5m234d
+//    6y5m234d37h
 func ParseDuration(s string) (Duration, error) {
 	var (
 		d   Duration
@@ -100,6 +104,8 @@ func ParseDuration(s string) (Duration, error) {
 			d.Months = num
 		case 'd':
 			d.Days = num
+		case 'h':
+			d.Hours = num
 		}

 		s = s[1:]
@@ -127,5 +133,5 @@ func (d Duration) Type() string {

 // Zero returns true if the duration is empty (all values are set to zero).
 func (d Duration) Zero() bool {
-	return d.Years == 0 && d.Months == 0 && d.Days == 0
+	return d.Years == 0 && d.Months == 0 && d.Days == 0 && d.Hours == 0
 }
--- a/internal/restic/duration_test.go
+++ b/internal/restic/duration_test.go
@@ -13,15 +13,24 @@ func TestNextNumber(t *testing.T) {
 		rest  string
 		err   bool
 	}{
+		{
+			input: "12h", num: 12, rest: "h",
+		},
 		{
 			input: "3d", num: 3, rest: "d",
 		},
+		{
+			input: "4d9h", num: 4, rest: "d9h",
+		},
 		{
 			input: "7m5d", num: 7, rest: "m5d",
 		},
 		{
 			input: "-23y7m5d", num: -23, rest: "y7m5d",
 		},
+		{
+			input: "-13y5m11d12h", num: -13, rest: "y5m11d12h",
+		},
 		{
 			input: "  5d", num: 0, rest: "  5d", err: true,
 		},
@@ -55,10 +64,15 @@ func TestParseDuration(t *testing.T) {
 		d      Duration
 		output string
 	}{
+		{"9h", Duration{Hours: 9}, "9h"},
 		{"3d", Duration{Days: 3}, "3d"},
+		{"4d2h", Duration{Days: 4, Hours: 2}, "4d2h"},
 		{"7m5d", Duration{Months: 7, Days: 5}, "7m5d"},
+		{"6m4d8h", Duration{Months: 6, Days: 4, Hours: 8}, "6m4d8h"},
 		{"5d7m", Duration{Months: 7, Days: 5}, "7m5d"},
+		{"4h3d9m", Duration{Months: 9, Days: 3, Hours: 4}, "9m3d4h"},
 		{"-7m5d", Duration{Months: -7, Days: 5}, "-7m5d"},
+		{"1y4m-5d-3h", Duration{Years: 1, Months: 4, Days: -5, Hours: -3}, "1y4m-5d-3h"},
 		{"2y7m-5d", Duration{Years: 2, Months: 7, Days: -5}, "2y7m-5d"},
 	}

--- a/internal/restic/node.go
+++ b/internal/restic/node.go
@@ -135,7 +135,7 @@ func (node Node) GetExtendedAttribute(a string) []byte {
 }

 // CreateAt creates the node at the given path but does NOT restore node meta data.
-func (node *Node) CreateAt(ctx context.Context, path string, repo Repository, idx *HardlinkIndex) error {
+func (node *Node) CreateAt(ctx context.Context, path string, repo Repository) error {
 	debug.Log("create node %v at %v", node.Name, path)

 	switch node.Type {
@@ -144,7 +144,7 @@ func (node *Node) CreateAt(ctx context.Context, path string, repo Repository, id
 			return err
 		}
 	case "file":
-		if err := node.createFileAt(ctx, path, repo, idx); err != nil {
+		if err := node.createFileAt(ctx, path, repo); err != nil {
 			return err
 		}
 	case "symlink":
@@ -259,18 +259,7 @@ func (node Node) createDirAt(path string) error {
 	return nil
 }

-func (node Node) createFileAt(ctx context.Context, path string, repo Repository, idx *HardlinkIndex) error {
-	if node.Links > 1 && idx.Has(node.Inode, node.DeviceID) {
-		if err := fs.Remove(path); !os.IsNotExist(err) {
-			return errors.Wrap(err, "RemoveCreateHardlink")
-		}
-		err := fs.Link(idx.GetFilename(node.Inode, node.DeviceID), path)
-		if err != nil {
-			return errors.Wrap(err, "CreateHardlink")
-		}
-		return nil
-	}
-
+func (node Node) createFileAt(ctx context.Context, path string, repo Repository) error {
 	f, err := fs.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_TRUNC, 0600)
 	if err != nil {
 		return errors.Wrap(err, "OpenFile")
@@ -287,10 +276,6 @@ func (node Node) createFileAt(ctx context.Context, path string, repo Repository,
 		return errors.Wrap(closeErr, "Close")
 	}

-	if node.Links > 1 {
-		idx.Add(node.Inode, node.DeviceID, path)
-	}
-
 	return nil
 }

--- a/internal/restic/node_test.go
+++ b/internal/restic/node_test.go
@@ -177,11 +177,9 @@ func TestNodeRestoreAt(t *testing.T) {
 		}
 	}()

-	idx := restic.NewHardlinkIndex()
-
 	for _, test := range nodeTests {
 		nodePath := filepath.Join(tempdir, test.Name)
-		rtest.OK(t, test.CreateAt(context.TODO(), nodePath, nil, idx))
+		rtest.OK(t, test.CreateAt(context.TODO(), nodePath, nil))
 		rtest.OK(t, test.RestoreMetadata(nodePath))

 		if test.Type == "symlink" && runtime.GOOS == "windows" {
--- a/internal/restic/snapshot_policy.go
+++ b/internal/restic/snapshot_policy.go
@@ -196,7 +196,7 @@ func ApplyPolicy(list Snapshots, p ExpirePolicy) (keep, remove Snapshots, reason

 		// If the timestamp of the snapshot is within the range, then keep it.
 		if !p.Within.Zero() {
-			t := latest.AddDate(-p.Within.Years, -p.Within.Months, -p.Within.Days)
+			t := latest.AddDate(-p.Within.Years, -p.Within.Months, -p.Within.Days).Add(time.Hour * time.Duration(-p.Within.Hours))
 			if cur.Time.After(t) {
 				keepSnap = true
 				keepSnapReasons = append(keepSnapReasons, fmt.Sprintf("within %v", p.Within))
--- a/internal/restic/snapshot_policy_test.go
+++ b/internal/restic/snapshot_policy_test.go
@@ -225,6 +225,9 @@ func TestApplyPolicy(t *testing.T) {
 		{Within: parseDuration("1m")},
 		{Within: parseDuration("1m14d")},
 		{Within: parseDuration("1y1d1m")},
+		{Within: parseDuration("13d23h")},
+		{Within: parseDuration("2m2h")},
+		{Within: parseDuration("1y2m3d3h")},
 	}

 	for i, p := range tests {
--- a/internal/restic/testdata/policy_keep_snapshots_27
+++ b/internal/restic/testdata/policy_keep_snapshots_27
@@ -0,0 +1,150 @@
+{
+  "keep": [
+    {
+      "time": "2016-01-18T12:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-12T21:08:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-12T21:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-09T21:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-08T20:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-07T10:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-06T08:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-05T09:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T16:23:03Z",
+      "tree": null,
+      "paths": null
+    }
+  ],
+  "reasons": [
+    {
+      "snapshot": {
+        "time": "2016-01-18T12:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-12T21:08:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-12T21:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-09T21:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-08T20:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-07T10:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-06T08:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-05T09:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T16:23:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 13d23h"
+      ],
+      "counters": {}
+    }
+  ]
+}
--- a/internal/restic/testdata/policy_keep_snapshots_28
+++ b/internal/restic/testdata/policy_keep_snapshots_28
@@ -0,0 +1,374 @@
+{
+  "keep": [
+    {
+      "time": "2016-01-18T12:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-12T21:08:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-12T21:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-09T21:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-08T20:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-07T10:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-06T08:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-05T09:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T16:23:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T12:30:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T12:28:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T12:24:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T12:23:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T11:23:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-04T10:23:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-03T07:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-01T07:08:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-01T01:03:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2016-01-01T01:02:03Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2015-11-22T10:20:30Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2015-11-21T10:20:30Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2015-11-20T10:20:30Z",
+      "tree": null,
+      "paths": null
+    },
+    {
+      "time": "2015-11-18T10:20:30Z",
+      "tree": null,
+      "paths": null
+    }
+  ],
+  "reasons": [
+    {
+      "snapshot": {
+        "time": "2016-01-18T12:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-12T21:08:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-12T21:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-09T21:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-08T20:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-07T10:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-06T08:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-05T09:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T16:23:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T12:30:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T12:28:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T12:24:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T12:23:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T11:23:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-04T10:23:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-03T07:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-01T07:08:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-01T01:03:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2016-01-01T01:02:03Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2015-11-22T10:20:30Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2015-11-21T10:20:30Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2015-11-20T10:20:30Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    },
+    {
+      "snapshot": {
+        "time": "2015-11-18T10:20:30Z",
+        "tree": null,
+        "paths": null
+      },
+      "matches": [
+        "within 2m2h"
+      ],
+      "counters": {}
+    }
+  ]
+}
--- a/internal/restic/testdata/policy_keep_snapshots_29
+++ b/internal/restic/testdata/policy_keep_snapshots_29
--- a/internal/restic/testing.go
+++ b/internal/restic/testing.go
@@ -26,6 +26,7 @@ type fakeFileSystem struct {
 	duplication float32
 	buf         []byte
 	chunker     *chunker.Chunker
+	rand        *rand.Rand
 }

 // saveFile reads from rd and saves the blobs in the repository. The list of
@@ -87,7 +88,7 @@ func (fs *fakeFileSystem) treeIsKnown(tree *Tree) (bool, []byte, ID) {
 }

 func (fs *fakeFileSystem) blobIsKnown(id ID, t BlobType) bool {
-	if rand.Float32() < fs.duplication {
+	if fs.rand.Float32() < fs.duplication {
 		return false
 	}

@@ -175,6 +176,7 @@ func TestCreateSnapshot(t testing.TB, repo Repository, at time.Time, depth int,
 		repo:        repo,
 		knownBlobs:  NewIDSet(),
 		duplication: duplication,
+		rand:        rand.New(rand.NewSource(seed)),
 	}

 	treeID := fs.saveTree(context.TODO(), seed, depth)
--- a/internal/restorer/doc.go
+++ b/internal/restorer/doc.go
@@ -0,0 +1,33 @@
+// Package restorer contains code to restore data from a repository.
+//
+// The Restorer tries to keep the number of backend requests minimal. It does
+// this by downloading all required blobs of a pack file with a single backend
+// request and avoiding repeated downloads of the same pack. In addition,
+// several pack files are fetched concurrently.
+//
+// Here is high-level pseudo-code of the how the Restorer attempts to achieve
+// these goals:
+//
+//   while there are packs to process
+//     choose a pack to process                      [1]
+//     get the pack from the backend or cache        [2]
+//     write pack blobs to the files that need them  [3]
+//     if not all pack blobs were used
+//       cache the pack for future use               [4]
+//
+// Pack download and processing (steps [2] - [4]) runs on multiple concurrent
+// Goroutines. The Restorer runs all steps [2]-[4] sequentially on the same
+// Goroutine.
+//
+// Before a pack is downloaded (step [2]), the required space is "reserved" in
+// the pack cache. Actual download uses single backend request to get all
+// required pack blobs. This may download blobs that are not needed, but we
+// assume it'll still be faster than getting individual blobs.
+//
+// Target files are written (step [3]) in the "right" order, first file blob
+// first, then second, then third and so on. Blob write order implies that some
+// pack blobs may not be immediately used, i.e. they are "out of order" for
+// their respective target files. Packs with unused blobs are cached (step
+// [4]). The cache has capacity limit and may purge packs before they are fully
+// used, in which case the purged packs will need to be re-downloaded.
+package restorer
--- a/internal/restorer/filepacktraverser.go
+++ b/internal/restorer/filepacktraverser.go
@@ -0,0 +1,52 @@
+package restorer
+
+import (
+	"github.com/restic/restic/internal/errors"
+	"github.com/restic/restic/internal/restic"
+)
+
+type filePackTraverser struct {
+	lookup func(restic.ID, restic.BlobType) ([]restic.PackedBlob, bool)
+}
+
+// iterates over all remaining packs of the file
+func (t *filePackTraverser) forEachFilePack(file *fileInfo, fn func(packIdx int, packID restic.ID, packBlobs []restic.Blob) bool) error {
+	if len(file.blobs) == 0 {
+		return nil
+	}
+
+	getBlobPack := func(blobID restic.ID) (restic.PackedBlob, error) {
+		packs, found := t.lookup(blobID, restic.DataBlob)
+		if !found {
+			return restic.PackedBlob{}, errors.Errorf("Unknown blob %s", blobID.String())
+		}
+		// TODO which pack to use if multiple packs have the blob?
+		// MUST return the same pack for the same blob during the same execution
+		return packs[0], nil
+	}
+
+	var prevPackID restic.ID
+	var prevPackBlobs []restic.Blob
+	packIdx := 0
+	for _, blobID := range file.blobs {
+		packedBlob, err := getBlobPack(blobID)
+		if err != nil {
+			return err
+		}
+		if !prevPackID.IsNull() && prevPackID != packedBlob.PackID {
+			if !fn(packIdx, prevPackID, prevPackBlobs) {
+				return nil
+			}
+			packIdx++
+		}
+		if prevPackID != packedBlob.PackID {
+			prevPackID = packedBlob.PackID
+			prevPackBlobs = make([]restic.Blob, 0)
+		}
+		prevPackBlobs = append(prevPackBlobs, packedBlob.Blob)
+	}
+	if len(prevPackBlobs) > 0 {
+		fn(packIdx, prevPackID, prevPackBlobs)
+	}
+	return nil
+}
--- a/Show More
+++ b/Show More
Author	SHA1	Message	Date
Alexander Neumann	cd9b2295f1	Add version for 0.9.4	2019-01-06 21:40:32 +01:00
Alexander Neumann	a439cdeb05	Update manpages and auto-completion	2019-01-06 21:40:32 +01:00
Alexander Neumann	827f6d7b24	Generate CHANGELOG.md for 0.9.4	2019-01-06 21:40:31 +01:00
Alexander Neumann	77ab10d401	Create changelog dir for 0.9.4	2019-01-06 21:39:56 +01:00
Alexander Neumann	3b0ad2e368	release: Pull latest alpine image	2019-01-06 21:39:56 +01:00
Alexander Neumann	2996c110f1	Merge pull request 2017 from kylewlacy/fuse_default_permissions_option mount: Add default-permissions flag to set FUSE option	2019-01-06 21:19:27 +01:00
Alexander Neumann	4609b5c24d	Merge pull request 2094 from juergenhoetzel/password-command Add support for reading password from external command	2019-01-06 21:14:52 +01:00
Alexander Neumann	830511460a	mount: Turn on DefaultPermissions for --allow-other` This commit changes the logic slightly: checking the permissions in the fuse mount when nobody else besides the current user can access the fuse mount does not sense. The current user has access to the repo files in addition to the password, so they can access all data regardless of what the fuse mount does. Enabling `--allow-root` allows the root user to access the files in the fuse mount, for this user no permission checks will be done anyway. The code now enables `DefaultPermissions` automatically when `--allow-other` is set, it can be disabled with `--no-default-permissions` to restore the old behavior.	2019-01-06 20:55:49 +01:00
Alexander Neumann	0dc3648416	Add documentation for `--password-command`	2019-01-06 20:41:38 +01:00
Alexander Neumann	d71dba3788	Merge pull request 2130 from andreaso/docs-https-links Prefer https:// links in the documentation	2019-01-06 20:24:42 +01:00
Alexander Neumann	e482633943	Merge pull request 2137 from kevinawoo/patch-1 doc: Add more clarity to B2 application key	2019-01-06 20:24:00 +01:00
Alexander Neumann	900621051a	Merge pull request 2120 from turnkey-commerce/fix-doc-typo Fix a few typos in the FAQ doc.	2019-01-06 15:08:13 +01:00
Alexander Neumann	1f246c5309	Merge pull request 2090 from plumbeo/within-hours Increase the granularity of the "keep within" snapshot retention policy	2019-01-06 15:04:55 +01:00
Alexander Neumann	e40805b002	Merge pull request 2103 from cdhowie/master --one-file-system unsupported on Windows	2019-01-06 14:59:46 +01:00
Kevin Woo	d65bea1b2a	doc: Add more clarity to B2 application key	2019-01-05 20:05:11 -08:00
Andreas Olsson	3b68acf853	Prefer https:// links in the documentation All the changed urls are available by way of https://. Most of them already redirect.	2018-12-29 15:57:44 +01:00
James Culbertson	82a70643a2	Fix a few typos in the FAQ doc.	2018-12-18 08:47:52 -06:00
Alexander Neumann	0dd805421e	Merge pull request 2108 from mholt/noexitprint cmd: Don't exit if printing to stdout or stderr fails (closes #2102)	2018-12-15 21:34:18 +01:00
Alexander Neumann	16b82f4b1d	Merge pull request 2111 from andreaso/doc-moot-sudo Remove moot sudo usage from documentation	2018-12-15 21:33:51 +01:00
Andreas Olsson	7a6bfcd58c	Remove moot sudo usage from documentation That usage of sudo only applies to the echo command itself. It has no effect on the permissions of the stdout redirect.	2018-12-09 17:43:50 +01:00
Matthew Holt	de54618852	cmd: Don't exit if printing to stdout or stderr fails (closes #2102 )	2018-12-01 15:11:22 -07:00
Alexander Neumann	98526b8dbe	Merge pull request 2100 from restic/fix-1989 gs: Respect bandwidth limiting	2018-11-28 21:39:42 +01:00
Alexander Neumann	0083680d33	Merge pull request 2099 from restic/fix-1229 cache: Don't return an error for truncated files	2018-11-28 21:39:35 +01:00
Chris Howie	05222b7343	--one-file-system unsupported on Windows	2018-11-27 11:14:47 -05:00
Kyle Lacy	d4ff5b6bf4	Add changelog entry about "DefaultPermissions" change	2018-11-26 21:16:01 -08:00
Kyle Lacy	cf0883e16c	mount: Add "no-default-permissions" option This option restores the previous behavior of `mount` by disabling the "DefaultPermissions" FUSE option. This allows any user that can access the mountpoint to read any file from the snapshot. Normal FUSE rules apply, so `allow-root` or `allow-other` can be used to allow users besides the mounting user to access these files.	2018-11-26 21:16:01 -08:00
Kyle Lacy	a35a24b8b4	mount: Enable "DefaultPermissions" FUSE option by default This enforces the Unix permissions of the snapshot files within the mounted filesystem, which will only allow users to access snapshot files if they had access to the file outside of the snapshot.	2018-11-26 21:16:01 -08:00
Juergen Hoetzel	df7f72cdde	Add support for reading password from external command This allows reading the password from an password manager (like "pass"). Signed-off-by: Juergen Hoetzel <juergen@archlinux.org>	2018-11-26 20:18:31 +01:00
plumbeo	3edc723bf0	Update 'restic forget --keep-within' documentation and add changelog entry	2018-11-26 14:27:48 +01:00
plumbeo	71891b340c	Support time ranges expressed in hours in snapshot retention policies Make restic forget --keep-within accept time ranges measured in hours and choose accordingly which snapshots to keep and which to forget. Add relative tests.	2018-11-26 14:27:42 +01:00
Alexander Neumann	6f5c3e57f6	Update vendor/modules.txt	2018-11-25 19:22:57 +01:00
Alexander Neumann	56af0ce370	Add entry to changelog	2018-11-25 19:07:42 +01:00
Alexander Neumann	c9745cd47e	gs: Respect bandwidth limiting In `0dfdc11ed9`, accidentally we dropped using the provided http.RoundTripper, this commits adds it back. Closes #1989	2018-11-25 18:52:32 +01:00
Alexander Neumann	2434ab2106	Merge pull request 2098 from cdhowie/key-hinting Add key hinting (#2097)	2018-11-25 17:52:54 +01:00
Chris Howie	1688713400	Add key hinting (#2097 )	2018-11-25 09:13:18 -05:00
Alexander Neumann	00597284de	cache: Don't return an error for truncated files Closes #1229 Closes #1328	2018-11-25 14:18:02 +01:00
Alexander Neumann	6dc7cca597	Merge pull request 2086 from frebib/fix-2085 Allow --files-from multiple times	2018-11-25 13:21:29 +01:00
Alexander Neumann	d32c7c2aba	Rework changelog for #2085	2018-11-25 12:55:03 +01:00
Joe Groocock	09e9b74cbd	Allow --files-from multiple times Fixes https://github.com/restic/restic/issues/2085	2018-11-25 12:55:03 +01:00
Alexander Neumann	d53595e43c	Merge pull request 2088 from garrmcnu/stats-latest-host stats: Add shorthand flag `-H` for host name filter	2018-11-24 19:05:53 +01:00
Alexander Neumann	0de19cc87f	Merge pull request 2095 from plumbeo/parse-location Use local time for user-provided time of backup	2018-11-24 18:50:12 +01:00
plumbeo	2c9ec07d0b	Add changelog entry	2018-11-20 22:18:12 +01:00
plumbeo	a7971a3ece	Use local time for user-provided time of backup	2018-11-20 21:28:19 +01:00
Alexander Neumann	4ab0022da8	Add hint which OS are supported Closes #2016	2018-11-18 12:33:52 +01:00
Garry McNulty	4b3c054257	stats: Add shorthand flag `-H` for host name filter The default value of the `--host` flag was set to 'H' (the shorthand version of the flag), this caused the snapshot lookup to fail. Also add shorthand `-H` for `backup` command. Closes #2040	2018-11-14 21:58:31 +00:00
plumbeo	7486bfea5b	Extend custom Duration granularity to hours and add tests	2018-11-14 17:27:02 +01:00
Alexander Neumann	c8fc72364a	Merge pull request 2081 from restic/fix-2080 backup: Open repository before async status starts	2018-11-11 14:21:47 +01:00
Alexander Neumann	987ef2f4a9	Merge pull request 2082 from restic/fix-2079 dump: Use path instead of filepath	2018-11-11 14:21:37 +01:00
Alexander Neumann	5b95bb7059	Fix error message on Windows	2018-11-11 14:06:32 +01:00
Alexander Neumann	8471a359ee	dump: Use path instead of filepath Some time ago we changed the paths in the repo to always use a slash for separation, it seems we missed that the `dump` command still uses the `filepath` package, so on Windows backslashes are used. Closes #2079	2018-11-10 12:43:11 +01:00
Alexander Neumann	f9422ff4c7	backup: Open repository before async status starts Closes #2080	2018-11-10 12:38:29 +01:00
Alexander Neumann	c0572ca15f	backup: Add warning when patterns do not match any files	2018-11-03 14:51:30 +01:00
Alexander Neumann	a630d69e0c	Merge pull request 2070 from restic/display-local-time Display local time for all commands	2018-11-02 21:16:29 +01:00
Alexander Neumann	20bcd281a3	Add entry to changelog	2018-11-02 20:40:52 +01:00
Alexander Neumann	c012fccd22	Display local time for all commands	2018-11-02 20:36:15 +01:00
Alexander Neumann	920727dd34	Merge pull request #2068 from restic/prune-check-load-error prune: Correctly handle errors returned by Load()	2018-10-28 22:15:29 +01:00
Alexander Neumann	157d365894	Add entry to CHANGELOG	2018-10-28 21:38:04 +01:00
Alexander Neumann	bfa18ee8ec	DownloadAndHash: Check error returned by Load()	2018-10-28 21:28:56 +01:00
Alexander Neumann	890eebf151	Correct summary for #1978	2018-10-28 20:55:49 +01:00
Alexander Neumann	9310cd0cd6	Merge pull request #2056 from restic/recover-data Add new command 'recover'	2018-10-28 14:26:48 +01:00
Alexander Neumann	9f7ce7ce5a	Merge pull request #2066 from restic/fix-list-error rebuild-index/prune: Correctly handle error listing files	2018-10-28 14:26:40 +01:00
Alexander Neumann	0b600d6cef	index.New: Test various error conditions	2018-10-28 14:02:31 +01:00
Alexander Neumann	3ae2a79bdf	restic: Use local rand source instead of global one	2018-10-28 14:01:30 +01:00
Alexander Neumann	f7c0893f76	index: Add tests for error conditions	2018-10-28 13:16:36 +01:00
Alexander Neumann	c3de301fc8	index: Use interfaces instead of *repository.Repository	2018-10-28 12:58:34 +01:00
Alexander Neumann	944b446ac0	index: Cancel context for tests	2018-10-28 11:17:04 +01:00
Alexander Neumann	b096fc7abf	index: Correctly process errors listing all files This also removes the now unused `list` and `worker` packages.	2018-10-28 11:16:29 +01:00
Alexander Neumann	d10754e2b4	Merge pull request 2057 from re1/patch-1 Update install instructions for Arch Linux	2018-10-21 20:04:07 +02:00
Alexander Neumann	7ac683c360	rclone: Inject debug logger for HTTP	2018-10-21 19:58:40 +02:00
Markus	6caa9d38ac	Update install instructions for Arch Linux restic has been added to the official community repositories in august 2018.	2018-10-20 13:09:52 +00:00
Alexander Neumann	19fd0f101f	Merge pull request 2053 from andyjack/correct-doc-for-self-update Fix incorrect command name in long doc for self-update	2018-10-20 11:51:13 +02:00
Alexander Neumann	8c91c51d1b	Fix timestamps	2018-10-20 11:08:03 +02:00
Alexander Neumann	7e28bf7e97	Add 'recover' command	2018-10-20 11:08:03 +02:00
Alexander Neumann	43d6e426c8	Merge pull request 2054 from gliptak/patch-3 Correct typo	2018-10-20 11:04:49 +02:00
Alexander Neumann	26fc60e7cb	Merge pull request 2055 from gliptak/patch-4 Correct ineffassign	2018-10-20 11:04:32 +02:00
Gábor Lipták	e5d7879622	Correct ineffassign Signed-off-by: Gábor Lipták <gliptak@gmail.com>	2018-10-19 16:58:14 -04:00
Gábor Lipták	d2ee58f2e9	Correct typo	2018-10-18 21:30:14 -04:00
Andy Jack	3f25537a06	Fix incorrect command name in long doc for self-update	2018-10-18 18:45:31 -04:00
Alexander Neumann	d203ae37f4	Merge pull request 2050 from andreasnuesslein/patch-1 tiny formatting fix	2018-10-18 22:31:15 +02:00
Andreas Nüßlein	6eedd66c1a	tiny formatting fix I noticed this discrepancy when trying to parse the output of restic.	2018-10-17 09:41:11 +02:00
Alexander Neumann	e4b39ae553	Merge pull request 2044 from restic/fix-verify restore: Close files tested with --verify	2018-10-15 21:16:58 +02:00
Alexander Neumann	7cbcb6d318	Merge pull request 2043 from cgroschupp/patch-1 Add ssh binary to docker image to allow sftp repository	2018-10-14 21:43:04 +02:00
Alexander Neumann	c0fca3f50a	restore: Close files tested with --verify Before, the target files were opened once per blob and never closed, this commit fixes that.	2018-10-14 21:00:14 +02:00
Alexander Neumann	4c2072d875	Merge pull request 1719 from ifedorenko/mt-restorer new optimized multithreaded restore implementation	2018-10-14 20:29:16 +02:00
Christian Groschupp	92ecca1808	Add ssh binary to docker image to allow sftp repository	2018-10-14 19:56:02 +02:00
Alexander Neumann	7236635cc1	Merge pull request 2042 from restic/fix-2041 self-update: fix path to output file	2018-10-14 19:48:28 +02:00
Alexander Neumann	21a3486ebb	Rework changelog entry	2018-10-14 17:39:42 +02:00
Igor Fedorenko	bda8d7722e	restorer: Optimize empty file restore don't create fileInfo structs for empty files. this saves memory. this also avoids extra serial scan of all fileInfo, which should make restore faster and more consistent. Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	c2bcb764cd	restorer: Add a note on hardlink metadata Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	9e24154ec9	restore: significantly reduce memory footprint reworked restore error callback to use file location path instead of much heavier Node. this reduced restore memory usage by as much as 50% in some of my tests. Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	9f3ca97ee8	restore: Chang fileInfo to use snapshot location instead of target path * uses less memory as common prefix is only stored once * stepping stone for simpler error callback api, which will allow further memory footprint reduction Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	32d5ceba87	restore: Fix packcache capacity math with failed downloads Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Igor Fedorenko	e010f3b884	restore: Fix small memory leak in filesWriter, add tests Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:39:42 +02:00
Alexander Neumann	941202c119	restore: Rename things, match the rest of the project	2018-10-14 17:39:37 +02:00
Alexander Neumann	c021ad2334	restore: Move documentation to godoc	2018-10-14 17:39:35 +02:00
Igor Fedorenko	2b3420820b	go mod vendor Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Igor Fedorenko	da57302fca	restore: Removed legacy restore implementation Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Igor Fedorenko	1869930d95	restore: New optimized multithreaded implementation Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Igor Fedorenko	1213d8fef4	Cleanup: more realistic restorer test data setup Signed-off-by: Igor Fedorenko <igor@ifedorenko.com>	2018-10-14 17:37:30 +02:00
Alexander Neumann	a432b42c81	self-update: Check current version before download	2018-10-14 17:29:46 +02:00
Alexander Neumann	7d0f2eaf24	self-update: Use correct path to restic binary Closes #2041	2018-10-14 17:29:19 +02:00
Alexander Neumann	41a4d67d93	Merge pull request 1962 from mholt/lsjson ls: Stream output when using --json option	2018-10-14 15:51:38 +02:00
Alexander Neumann	afde60e433	Merge pull request #2039 from restic/fix-2038 snapshots: Don't print empty line for --compact	2018-10-13 21:24:26 +02:00
Alexander Neumann	d7baa67acb	snapshots: Don't print empty line for --compact Closes #2038	2018-10-13 20:17:34 +02:00
Alexander Neumann	167397c18c	Set development version for 0.9.3	2018-10-13 13:48:31 +02:00
Matthew Holt	9151eec24e	Add changelog entry	2018-08-19 09:14:58 -06:00
Matthew Holt	22475729ce	Select specific Node fields for listing	2018-08-19 09:03:47 -06:00
Matthew Holt	04c67d700d	ls: Stream output when using --json option	2018-08-19 00:18:43 -06:00
@@ -1 +1 @@
 .9.3
 .9.4