Set development version for 0.18.1

doc: Nitpicks on changelogs

CHANGELOG.md

@@ -1,5 +1,6 @@
 # Table of Contents
 
+* [Changelog for 0.18.1](#changelog-for-restic-0181-2025-09-21)
 * [Changelog for 0.18.0](#changelog-for-restic-0180-2025-03-27)
 * [Changelog for 0.17.3](#changelog-for-restic-0173-2024-11-08)
 * [Changelog for 0.17.2](#changelog-for-restic-0172-2024-10-27)
@@ -39,6 +40,106 @@
 * [Changelog for 0.6.0](#changelog-for-restic-060-2017-05-29)
 
 
+# Changelog for restic 0.18.1 (2025-09-21)
+The following sections list the changes in restic 0.18.1 relevant to
+restic users. The changes are ordered by importance.
+
+## Summary
+
+ * Fix #5324: Correctly handle `backup --stdin-filename` with directory paths
+ * Fix #5325: Accept `RESTIC_HOST` environment variable in `forget` command
+ * Fix #5342: Ignore "chmod not supported" errors when writing files
+ * Fix #5344: Ignore `EOPNOTSUPP` errors for extended attributes
+ * Fix #5421: Fix rare crash if directory is removed during backup
+ * Fix #5429: Stop retrying uploads when rest-server runs out of space
+ * Fix #5467: Improve handling of download retries in `check` command
+
+## Details
+
+ * Bugfix #5324: Correctly handle `backup --stdin-filename` with directory paths
+
+   In restic 0.18.0, the `backup` command failed if a filename that includes at
+   least a directory was passed to `--stdin-filename`. For example,
+   `--stdin-filename /foo/bar` resulted in the following error:
+
+   ```
+   Fatal: unable to save snapshot: open /foo: no such file or directory
+   ```
+
+   This has now been fixed.
+
+   https://github.com/restic/restic/issues/5324
+   https://github.com/restic/restic/pull/5356
+
+ * Bugfix #5325: Accept `RESTIC_HOST` environment variable in `forget` command
+
+   The `forget` command did not use the host name from the `RESTIC_HOST`
+   environment variable when filtering snapshots. This has now been fixed.
+
+   https://github.com/restic/restic/issues/5325
+   https://github.com/restic/restic/pull/5327
+
+ * Bugfix #5342: Ignore "chmod not supported" errors when writing files
+
+   Restic 0.18.0 introduced a bug that caused `chmod xxx: operation not supported`
+   errors to appear when writing to a local file repository that did not support
+   chmod (like CIFS or WebDAV mounted via FUSE). Restic now ignores those errors.
+
+   https://github.com/restic/restic/issues/5342
+
+ * Bugfix #5344: Ignore `EOPNOTSUPP` errors for extended attributes
+
+   Restic 0.18.0 added extended attribute support for NetBSD 10+, but not all
+   NetBSD filesystems support extended attributes. Other BSD systems can likewise
+   return `EOPNOTSUPP`, so restic now ignores these errors.
+
+   https://github.com/restic/restic/issues/5344
+
+ * Bugfix #5421: Fix rare crash if directory is removed during backup
+
+   In restic 0.18.0, the `backup` command could crash if a directory was removed
+   between reading its metadata and listing its directory content. This has now
+   been fixed.
+
+   https://github.com/restic/restic/pull/5421
+
+ * Bugfix #5429: Stop retrying uploads when rest-server runs out of space
+
+   When rest-server returns a `507 Insufficient Storage` error, it indicates that
+   no more storage capacity is available. Restic now correctly stops retrying
+   uploads in this case.
+
+   https://github.com/restic/restic/issues/5429
+   https://github.com/restic/restic/pull/5452
+
+ * Bugfix #5467: Improve handling of download retries in `check` command
+
+   In very rare cases, the `check` command could unnecessarily report repository
+   damage if the backend returned incomplete, corrupted data on the first download
+   try which is afterwards resolved by a download retry.
+
+   This could result in an error output like the following:
+
+   ```
+   Load(<data/34567890ab>, 33918928, 0) returned error, retrying after 871.35598ms: readFull: unexpected EOF
+   Load(<data/34567890ab>, 33918928, 0) operation successful after 1 retries
+   check successful on second attempt, original error pack 34567890ab[...] contains 6 errors: [blob 12345678[...]: decrypting blob <data/12345678> from 34567890 failed: ciphertext verification failed ...]
+   [...]
+   Fatal: repository contains errors
+   ```
+
+   This fix only applies to a very specific case where the log shows `operation
+   successful after 1 retries` followed by a `check successful on second attempt,
+   original error` that only reports `ciphertext verification failed` errors in the
+   pack file. If any other errors are reported in the pack file, then the
+   repository still has to be considered as damaged.
+
+   Now, only the check result of the last download retry is reported as intended.
+
+   https://github.com/restic/restic/issues/5467
+   https://github.com/restic/restic/pull/5495
+
+
 # Changelog for restic 0.18.0 (2025-03-27)
 The following sections list the changes in restic 0.18.0 relevant to
 restic users. The changes are ordered by importance.

VERSION

@@ -1 +1 @@
-0.18.0-dev
+0.18.1-dev

changelog/0.18.1_2025-09-21/issue-5324

@@ -1,14 +1,14 @@
-Bugfix: Correctly handle `backup --stdin-filename` with directories
+Bugfix: Correctly handle `backup --stdin-filename` with directory paths
 
 In restic 0.18.0, the `backup` command failed if a filename that includes
-a least a directory was passed to `--stdin-filename`. For example,
+at least a directory was passed to `--stdin-filename`. For example,
 `--stdin-filename /foo/bar` resulted in the following error:
 
 ```
 Fatal: unable to save snapshot: open /foo: no such file or directory
 ```
 
-This has been fixed now.
+This has now been fixed.
 
 https://github.com/restic/restic/issues/5324
 https://github.com/restic/restic/pull/5356

changelog/0.18.1_2025-09-21/issue-5325

@@ -1,7 +1,7 @@
-Bugfix: Correctly handle `RESTIC_HOST` in `forget` command
+Bugfix: Accept `RESTIC_HOST` environment variable in `forget` command
 
 The `forget` command did not use the host name from the `RESTIC_HOST`
-environment variable. This has been fixed.
+environment variable when filtering snapshots. This has now been fixed.
 
 https://github.com/restic/restic/issues/5325
 https://github.com/restic/restic/pull/5327

changelog/0.18.1_2025-09-21/issue-5342

@@ -1,6 +1,6 @@
 Bugfix: Ignore "chmod not supported" errors when writing files
 
-Restic 0.18.0 introduced a bug that caused "chmod xxx: operation not supported"
+Restic 0.18.0 introduced a bug that caused `chmod xxx: operation not supported`
 errors to appear when writing to a local file repository that did not support
 chmod (like CIFS or WebDAV mounted via FUSE). Restic now ignores those errors.
 

changelog/0.18.1_2025-09-21/issue-5344

@@ -0,0 +1,7 @@
+Bugfix: Ignore `EOPNOTSUPP` errors for extended attributes
+
+Restic 0.18.0 added extended attribute support for NetBSD 10+, but not all
+NetBSD filesystems support extended attributes. Other BSD systems can
+likewise return `EOPNOTSUPP`, so restic now ignores these errors.
+
+https://github.com/restic/restic/issues/5344

changelog/0.18.1_2025-09-21/issue-5429

@@ -0,0 +1,8 @@
+Bugfix: Stop retrying uploads when rest-server runs out of space
+
+When rest-server returns a `507 Insufficient Storage` error, it indicates
+that no more storage capacity is available. Restic now correctly stops
+retrying uploads in this case.
+
+https://github.com/restic/restic/issues/5429
+https://github.com/restic/restic/pull/5452

changelog/0.18.1_2025-09-21/issue-5467

@@ -0,0 +1,27 @@
+Bugfix: Improve handling of download retries in `check` command
+
+In very rare cases, the `check` command could unnecessarily report repository
+damage if the backend returned incomplete, corrupted data on the first download
+try which is afterwards resolved by a download retry.
+
+This could result in an error output like the following:
+
+```
+Load(<data/34567890ab>, 33918928, 0) returned error, retrying after 871.35598ms: readFull: unexpected EOF
+Load(<data/34567890ab>, 33918928, 0) operation successful after 1 retries
+check successful on second attempt, original error pack 34567890ab[...] contains 6 errors: [blob 12345678[...]: decrypting blob <data/12345678> from 34567890 failed: ciphertext verification failed ...]
+[...]
+Fatal: repository contains errors
+```
+
+This fix only applies to a very specific case where the log shows
+`operation successful after 1 retries` followed by a
+`check successful on second attempt, original error` that only reports
+`ciphertext verification failed` errors in the pack file. If any other errors
+are reported in the pack file, then the repository still has to be considered
+as damaged.
+
+Now, only the check result of the last download retry is reported as intended.
+
+https://github.com/restic/restic/issues/5467
+https://github.com/restic/restic/pull/5495

changelog/0.18.1_2025-09-21/pull-5421

@@ -1,8 +1,7 @@
 Bugfix: Fix rare crash if directory is removed during backup
 
-In restic 0.18.0, the `backup` command could crash if a directory is removed
-inbetween reading its metadata and listing its directory content.
-
-This has been fixed.
+In restic 0.18.0, the `backup` command could crash if a directory was removed
+between reading its metadata and listing its directory content. This has now
+been fixed.
 
 https://github.com/restic/restic/pull/5421

changelog/unreleased/issue-4728

@@ -1,7 +0,0 @@
-Enhancement: Added support for zstd compression levels `fastest` and `better`
-
-Restic now supports the zstd compression modes `fastest` and `better`. Set the
-environment variable `RESTIC_COMPRESSION` to `fastest` or `better` to use these
-compression levels. This can also be set with the `--compression` flag.
-
-https://github.com/restic/restic/issues/4728

changelog/unreleased/issue-4868

@@ -1,14 +0,0 @@
-Enhancement: Include repository id in filesystem name used by `mount`
-
-The filesystem created by restic's `mount` command now includes the repository
-id in the filesystem name. The repository id is printed by restic when opening
-a repository or can be looked up using `restic cat config`.
-
-```
-[restic-user@hostname restic]$ df ./test-mount/
-Filesystem        1K-blocks  Used Available Use% Mounted on
-restic:d3b07384d1         0     0         0    - /mnt/my-restic-repo
-```
-
-https://github.com/restic/restic/issues/4868
-https://github.com/restic/restic/pull/5243

changelog/unreleased/issue-5233

@@ -1,8 +0,0 @@
-Bugfix: forget command returns exit code 3 on partial removal of snapshots
-
-The `forget` command now returns exit code 3 when it fails to remove one or
-more snapshots. Previously, it returned exit code 0, which could lead to
-confusion if the command was used in a script.
-
-https://github.com/restic/restic/issues/5233
-https://github.com/restic/restic/pull/5322

changelog/unreleased/issue-5344

@@ -1,7 +0,0 @@
-Bugfix: Ignore EOPNOTSUPP as an error for xattr
-
-Restic 0.18.0 added xattr support for NetBSD 10+, but not all NetBSD
-filesystems support xattrs.  Other BSD systems can likewise return
-EOPNOTSUPP, so restic now simply ignores EOPNOTSUPP errors for xattrs.
-
-https://github.com/restic/restic/issues/5344

changelog/unreleased/issue-5354

@@ -1,12 +0,0 @@
-Bugfix: Allow use of rclone/sftp backend when running restic in background
-
-When starting restic in the background, this could result in unexpected behavior
-when using the rclone or sftp backend.
-
-For example running `restic -r rclone:./example --insecure-no-password init &`
-could cause the calling `bash` shell to exit unexpectedly.
-
-This has been fixed.
-
-https://github.com/restic/restic/issues/5354
-https://github.com/restic/restic/pull/5358

changelog/unreleased/issue-5429

@@ -1,8 +0,0 @@
-Bugfix: do not retry if rest-server runs out of space
-
-Rest-server return error `507 Insufficient Storage` if no more storage
-capacity is available at the server. Restic now no longer retries uploads
-in this case.
-
-https://github.com/restic/restic/issues/5429
-https://github.com/restic/restic/pull/5452

cmd/restic/cmd_check.go

@@ -425,8 +425,7 @@ func runCheck(ctx context.Context, opts CheckOptions, gopts GlobalOptions, args
 				subsetSize = repoSize
 			}
 			packs = selectRandomPacksByFileSize(chkr.GetPacks(), subsetSize, repoSize)
-			percentage := float64(subsetSize) / float64(repoSize) * 100.0
-			printer.P("read %d bytes (%.1f%%) of data packs\n", subsetSize, percentage)
+			printer.P("read %d bytes of data packs\n", subsetSize)
 		}
 		if packs == nil {
 			return summary, errors.Fatal("internal error: failed to select packs to check")

cmd/restic/cmd_forget.go

@@ -41,7 +41,6 @@ EXIT STATUS
 
 Exit status is 0 if the command was successful.
 Exit status is 1 if there was any error.
-Exit status is 3 if there was an error removing one or more snapshots.
 Exit status is 10 if the repository does not exist.
 Exit status is 11 if the repository is already locked.
 Exit status is 12 if the password is incorrect.
@@ -63,7 +62,6 @@ Exit status is 12 if the password is incorrect.
 type ForgetPolicyCount int
 
 var ErrNegativePolicyCount = errors.New("negative values not allowed, use 'unlimited' instead")
-var ErrFailedToRemoveOneOrMoreSnapshots = errors.New("failed to remove one or more snapshots")
 
 func (c *ForgetPolicyCount) Set(s string) error {
 	switch s {
@@ -307,15 +305,12 @@ func runForget(ctx context.Context, opts ForgetOptions, pruneOptions PruneOption
 		return ctx.Err()
 	}
 
-	// these are the snapshots that failed to be removed
-	failedSnIDs := restic.NewIDSet()
 	if len(removeSnIDs) > 0 {
 		if !opts.DryRun {
 			bar := printer.NewCounter("files deleted")
 			err := restic.ParallelRemove(ctx, repo, removeSnIDs, restic.WriteableSnapshotFile, func(id restic.ID, err error) error {
 				if err != nil {
 					printer.E("unable to remove %v/%v from the repository\n", restic.SnapshotFile, id)
-					failedSnIDs.Insert(id)
 				} else {
 					printer.VV("removed %v/%v\n", restic.SnapshotFile, id)
 				}
@@ -337,10 +332,6 @@ func runForget(ctx context.Context, opts ForgetOptions, pruneOptions PruneOption
 		}
 	}
 
-	if len(failedSnIDs) > 0 {
-		return ErrFailedToRemoveOneOrMoreSnapshots
-	}
-
 	if len(removeSnIDs) > 0 && opts.Prune {
 		if opts.DryRun {
 			printer.P("%d snapshots would be removed, running prune dry run\n", len(removeSnIDs))

cmd/restic/cmd_mount.go

@@ -5,7 +5,6 @@ package main
 
 import (
 	"context"
-	"fmt"
 	"os"
 	"strings"
 	"time"
@@ -149,11 +148,9 @@ func runMount(ctx context.Context, opts MountOptions, gopts GlobalOptions, args
 		return err
 	}
 
-	fuseMountName := fmt.Sprintf("restic:%s", repo.Config().ID[:10])
-
 	mountOptions := []systemFuse.MountOption{
 		systemFuse.ReadOnly(),
-		systemFuse.FSName(fuseMountName),
+		systemFuse.FSName("restic"),
 		systemFuse.MaxReadahead(128 * 1024),
 	}
 

cmd/restic/global.go

@@ -46,7 +46,7 @@ import (
 // to a missing backend storage location or config file
 var ErrNoRepository = errors.New("repository does not exist")
 
-var version = "0.18.0-dev (compiled manually)"
+var version = "0.18.1-dev (compiled manually)"
 
 // TimeFormat is the format used for all timestamps printed by restic.
 const TimeFormat = "2006-01-02 15:04:05"
@@ -114,7 +114,7 @@ func (opts *GlobalOptions) AddFlags(f *pflag.FlagSet) {
 	f.BoolVar(&opts.InsecureNoPassword, "insecure-no-password", false, "use an empty password for the repository, must be passed to every restic command (insecure)")
 	f.BoolVar(&opts.InsecureTLS, "insecure-tls", false, "skip TLS certificate verification when connecting to the repository (insecure)")
 	f.BoolVar(&opts.CleanupCache, "cleanup-cache", false, "auto remove old cache directories")
-	f.Var(&opts.Compression, "compression", "compression mode (only available for repository format version 2), one of (auto|off|fastest|better|max) (default: $RESTIC_COMPRESSION)")
+	f.Var(&opts.Compression, "compression", "compression mode (only available for repository format version 2), one of (auto|off|max) (default: $RESTIC_COMPRESSION)")
 	f.BoolVar(&opts.NoExtraVerify, "no-extra-verify", false, "skip additional verification of data before upload (see documentation)")
 	f.IntVar(&opts.Limits.UploadKb, "limit-upload", 0, "limits uploads to a maximum `rate` in KiB/s. (default: unlimited)")
 	f.IntVar(&opts.Limits.DownloadKb, "limit-download", 0, "limits downloads to a maximum `rate` in KiB/s. (default: unlimited)")

cmd/restic/main.go

@@ -206,8 +206,6 @@ func main() {
 		exitCode = 0
 	case err == ErrInvalidSourceData:
 		exitCode = 3
-	case errors.Is(err, ErrFailedToRemoveOneOrMoreSnapshots):
-		exitCode = 3
 	case errors.Is(err, ErrNoRepository):
 		exitCode = 10
 	case restic.IsAlreadyLocked(err):

doc/047_tuning_backup_parameters.rst

@@ -55,10 +55,11 @@ Compression
 ===========
 
 For a repository using at least repository format version 2, you can configure how data
-is compressed with the option ``--compression``. It can be set to ``off``, ``fastest``,
-``auto`` (default), ``better``, or ``max``. Each setting uses more CPU but less bandwidth
-and storage space. This setting is only applied for the single run of restic, but can also be
-set via the environment variable ``RESTIC_COMPRESSION``.
+is compressed with the option ``--compression``. It can be set to ``auto`` (the default,
+which will compress very fast), ``max`` (which will trade backup speed and CPU usage for
+slightly better compression), or ``off`` (which disables compression). Each setting is
+only applied for the single run of restic. The option can also be set via the environment
+variable ``RESTIC_COMPRESSION``.
 
 
 Data Verification

doc/080_examples.rst

@@ -353,72 +353,3 @@ system.
 
    root@a3e580b6369d:/# sudo -u restic /home/restic/bin/restic --exclude={/dev,/media,/mnt,/proc,/run,/sys,/tmp,/var/tmp} -r /tmp backup /
 
-
-***********************************************************
-Back up to an internal repository server over an SSH tunnel
-***********************************************************
-
-Idea
-====
-
-The idea is to run `REST-server <https://github.com/restic/rest-server>`__ on
-an internal host as the repository server and then back up to it from a remote
-restic client through a reverse SSH tunnel.
-
-With this approach, you do not need to publicly expose the repository server
-to which the backups are sent, as the restic client can instead connect to it
-through the SSH tunnel.
-
-An example use case for this method would be to create backups of a server,
-e.g. a VPS in the cloud, to a repository stored on your local computer.
-
-Running a local repository server
-=================================
-
-On the internal host, download and run the latest `release <https://github.com/restic/rest-server/releases>`__
-of REST-server to act as the repository server. In this example we are using
-the ``--no-auth`` option to not require authentication when connecting to it:
-
-.. code-block:: console
-
-   rest-server --path /path/to/repo --no-auth
-
-.. note:: REST-server by default listens on all network interfaces and port
-          ``8000``.
-
-Creating a reverse SSH tunnel
-=============================
-
-On the repository server (the internal host), use ``ssh -R`` to create what's
-called a "reverse" SSH tunnel that listens for connections on the *remote* side
-and forwards these back through the tunnel to the *local* side:
-
-.. code-block:: console
-
-   ssh -R 8000:localhost:8000 user@server
-
-.. note:: In this example, ``localhost`` refers to the local repository server,
-          and ``server`` refers to the remote system where restic is to be run.
-
-Running restic on the remote system
-===================================
-
-Now that the SSH session and tunnel is established, run restic on the remote
-system as usual, but with a repository URL that targets that system's side of
-the SSH tunnel, in this example ``localhost:8000``.
-
-This will make restic on the remote system connect to port ``8000`` on its
-``localhost``, where the SSH tunnel is listening, after which the connection
-is forwarded through the tunnel and finally reaches ``localhost:8000`` on the
-local side where REST-server is listening and acting as the repository server.
-
-To initialize the repository:
-
-.. code-block:: console
-
-   restic -r rest:http://localhost:8000/ init
-
-You can then use standard restic commands such as ``backup``, ``snapshots`` and
-``restore`` with the same repository URL and other options as usual.
-
-.. tip:: The tunnel will be active for the duration of the SSH session.

internal/backend/util/foreground_sysv.go

@@ -0,0 +1,28 @@
+//go:build aix || solaris
+// +build aix solaris
+
+package util
+
+import (
+	"os/exec"
+	"syscall"
+
+	"github.com/restic/restic/internal/errors"
+)
+
+func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
+	// run the command in its own process group so that SIGINT
+	// is not sent to it.
+	cmd.SysProcAttr = &syscall.SysProcAttr{
+		Setpgid: true,
+	}
+
+	// start the process
+	err = cmd.Start()
+	if err != nil {
+		return nil, errors.Wrap(err, "cmd.Start")
+	}
+
+	bg = func() error { return nil }
+	return bg, nil
+}

internal/backend/util/foreground_unix.go

@@ -1,4 +1,5 @@
-//go:build unix
+//go:build !aix && !solaris && !windows
+// +build !aix,!solaris,!windows
 
 package util
 
@@ -9,45 +10,35 @@ import (
 
 	"github.com/restic/restic/internal/debug"
 	"github.com/restic/restic/internal/errors"
-	"github.com/restic/restic/internal/ui/termstatus"
 
 	"golang.org/x/sys/unix"
 )
 
-func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
-	// run the command in its own process group
-	// this ensures that sending ctrl-c to restic will not immediately stop the backend process.
-	cmd.SysProcAttr = &unix.SysProcAttr{
-		Setpgid: true,
-	}
+func tcsetpgrp(fd int, pid int) error {
+	// IoctlSetPointerInt silently casts to int32 internally,
+	// so this assumes pid fits in 31 bits.
+	return unix.IoctlSetPointerInt(fd, unix.TIOCSPGRP, pid)
+}
 
+func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
 	// open the TTY, we need the file descriptor
 	tty, err := os.OpenFile("/dev/tty", os.O_RDWR, 0)
 	if err != nil {
 		debug.Log("unable to open tty: %v", err)
-		return startFallback(cmd)
-	}
-
-	// only move child process to foreground if restic is in the foreground
-	prev, err := termstatus.Tcgetpgrp(int(tty.Fd()))
-	if err != nil {
-		_ = tty.Close()
-		return nil, err
-	}
-
-	self := termstatus.Getpgrp()
-	if prev != self {
-		debug.Log("restic is not controlling the tty; err = %v", err)
-		if err := tty.Close(); err != nil {
-			return nil, err
+		bg = func() error {
+			return nil
 		}
-		return startFallback(cmd)
+		return bg, cmd.Start()
 	}
 
-	// Prevent getting suspended when interacting with the tty
 	signal.Ignore(unix.SIGTTIN)
 	signal.Ignore(unix.SIGTTOU)
 
+	// run the command in its own process group
+	cmd.SysProcAttr = &unix.SysProcAttr{
+		Setpgid: true,
+	}
+
 	// start the process
 	err = cmd.Start()
 	if err != nil {
@@ -56,7 +47,8 @@ func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
 	}
 
 	// move the command's process group into the foreground
-	err = termstatus.Tcsetpgrp(int(tty.Fd()), cmd.Process.Pid)
+	prev := unix.Getpgrp()
+	err = tcsetpgrp(int(tty.Fd()), cmd.Process.Pid)
 	if err != nil {
 		_ = tty.Close()
 		return nil, err
@@ -67,7 +59,7 @@ func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
 		signal.Reset(unix.SIGTTOU)
 
 		// reset the foreground process group
-		err = termstatus.Tcsetpgrp(int(tty.Fd()), prev)
+		err = tcsetpgrp(int(tty.Fd()), prev)
 		if err != nil {
 			_ = tty.Close()
 			return err
@@ -78,11 +70,3 @@ func startForeground(cmd *exec.Cmd) (bg func() error, err error) {
 
 	return bg, nil
 }
-
-func startFallback(cmd *exec.Cmd) (bg func() error, err error) {
-	bg = func() error {
-		return nil
-	}
-
-	return bg, cmd.Start()
-}

internal/checker/checker_test.go

@@ -293,7 +293,7 @@ type errorBackend struct {
 func (b errorBackend) Load(ctx context.Context, h backend.Handle, length int, offset int64, consumer func(rd io.Reader) error) error {
 	return b.Backend.Load(ctx, h, length, offset, func(rd io.Reader) error {
 		if b.ProduceErrors {
-			return consumer(errorReadCloser{rd})
+			return consumer(errorReadCloser{Reader: rd})
 		}
 		return consumer(rd)
 	})
@@ -301,12 +301,21 @@ func (b errorBackend) Load(ctx context.Context, h backend.Handle, length int, of
 
 type errorReadCloser struct {
 	io.Reader
+	shortenBy      int
+	maxErrorOffset int // if 0, the error can be injected at any offset
 }
 
 func (erd errorReadCloser) Read(p []byte) (int, error) {
 	n, err := erd.Reader.Read(p)
 	if n > 0 {
-		induceError(p[:n])
+		maxOffset := n
+		if erd.maxErrorOffset > 0 {
+			maxOffset = min(erd.maxErrorOffset, maxOffset)
+		}
+		induceError(p[:maxOffset])
+	}
+	if n > erd.shortenBy {
+		n -= erd.shortenBy
 	}
 	return n, err
 }
@@ -320,17 +329,26 @@ func induceError(data []byte) {
 // errorOnceBackend randomly modifies data when reading a file for the first time.
 type errorOnceBackend struct {
 	backend.Backend
-	m sync.Map
+	m              sync.Map
+	shortenBy      int
+	maxErrorOffset int
 }
 
 func (b *errorOnceBackend) Load(ctx context.Context, h backend.Handle, length int, offset int64, consumer func(rd io.Reader) error) error {
 	_, isRetry := b.m.LoadOrStore(h, struct{}{})
-	return b.Backend.Load(ctx, h, length, offset, func(rd io.Reader) error {
+	err := b.Backend.Load(ctx, h, length, offset, func(rd io.Reader) error {
 		if !isRetry && h.Type != restic.ConfigFile {
-			return consumer(errorReadCloser{rd})
+			return consumer(errorReadCloser{Reader: rd, shortenBy: b.shortenBy, maxErrorOffset: b.maxErrorOffset})
 		}
 		return consumer(rd)
 	})
+	if err == nil {
+		return nil
+	}
+	// retry if the consumer returned an error
+	return b.Backend.Load(ctx, h, length, offset, func(rd io.Reader) error {
+		return consumer(rd)
+	})
 }
 
 func TestCheckerModifiedData(t *testing.T) {
@@ -368,6 +386,15 @@ func TestCheckerModifiedData(t *testing.T) {
 				}
 			},
 		},
+		{
+			// ignore if a backend returns incomplete garbled data on the first try
+			"corruptPartialOnceBackend",
+			&errorOnceBackend{Backend: be, shortenBy: 10, maxErrorOffset: 100},
+			func() {},
+			func(t *testing.T, err error) {
+				test.Assert(t, err == nil, "unexpected error found, got %v", err)
+			},
+		},
 	} {
 		t.Run(test.name, func(t *testing.T) {
 			checkRepo := repository.TestOpenBackend(t, test.be)

internal/repository/check.go

@@ -88,10 +88,14 @@ func checkPackInner(ctx context.Context, r *Repository, id restic.ID, blobs []re
 	// calculate hash on-the-fly while reading the pack and capture pack header
 	var hash restic.ID
 	var hdrBuf []byte
+	// must use a separate slice from `errs` here as we're only interested in the last retry
+	var blobErrors []error
 	h := backend.Handle{Type: backend.PackFile, Name: id.String()}
 	err := r.be.Load(ctx, h, int(size), 0, func(rd io.Reader) error {
 		hrd := hashing.NewReader(rd, sha256.New())
 		bufRd.Reset(hrd)
+		// reset blob errors for each retry
+		blobErrors = nil
 
 		it := newPackBlobIterator(id, newBufReader(bufRd), 0, blobs, r.Key(), dec)
 		for {
@@ -108,7 +112,7 @@ func checkPackInner(ctx context.Context, r *Repository, id restic.ID, blobs []re
 			debug.Log("  check blob %v: %v", val.Handle.ID, val.Handle)
 			if val.Err != nil {
 				debug.Log("  error verifying blob %v: %v", val.Handle.ID, val.Err)
-				errs = append(errs, errors.Errorf("blob %v: %v", val.Handle.ID, val.Err))
+				blobErrors = append(blobErrors, errors.Errorf("blob %v: %v", val.Handle.ID, val.Err))
 			}
 		}
 
@@ -134,6 +138,7 @@ func checkPackInner(ctx context.Context, r *Repository, id restic.ID, blobs []re
 		hash = restic.IDFromHash(hrd.Sum(nil))
 		return nil
 	})
+	errs = append(errs, blobErrors...)
 	if err != nil {
 		var e *partialReadError
 		isPartialReadError := errors.As(err, &e)

internal/repository/repository.go

@@ -73,9 +73,7 @@ const (
 	CompressionAuto    CompressionMode = 0
 	CompressionOff     CompressionMode = 1
 	CompressionMax     CompressionMode = 2
-	CompressionFastest CompressionMode = 3
-	CompressionBetter  CompressionMode = 4
-	CompressionInvalid CompressionMode = 5
+	CompressionInvalid CompressionMode = 3
 )
 
 // Set implements the method needed for pflag command flag parsing.
@@ -87,13 +85,9 @@ func (c *CompressionMode) Set(s string) error {
 		*c = CompressionOff
 	case "max":
 		*c = CompressionMax
-	case "fastest":
-		*c = CompressionFastest
-	case "better":
-		*c = CompressionBetter
 	default:
 		*c = CompressionInvalid
-		return fmt.Errorf("invalid compression mode %q, must be one of (auto|off|fastest|better|max)", s)
+		return fmt.Errorf("invalid compression mode %q, must be one of (auto|off|max)", s)
 	}
 
 	return nil
@@ -107,10 +101,6 @@ func (c *CompressionMode) String() string {
 		return "off"
 	case CompressionMax:
 		return "max"
-	case CompressionFastest:
-		return "fastest"
-	case CompressionBetter:
-		return "better"
 	default:
 		return "invalid"
 	}
@@ -315,17 +305,9 @@ func (r *Repository) loadBlob(ctx context.Context, blobs []restic.PackedBlob, bu
 
 func (r *Repository) getZstdEncoder() *zstd.Encoder {
 	r.allocEnc.Do(func() {
-
-		var level zstd.EncoderLevel
-		switch r.opts.Compression {
-		case CompressionFastest:
-			level = zstd.SpeedFastest
-		case CompressionBetter:
-			level = zstd.SpeedBetterCompression
-		case CompressionMax:
+		level := zstd.SpeedDefault
+		if r.opts.Compression == CompressionMax {
 			level = zstd.SpeedBestCompression
-		default:
-			level = zstd.SpeedDefault
 		}
 
 		opts := []zstd.EOption{

internal/ui/termstatus/background.go

@@ -1,3 +1,6 @@
+//go:build !linux
+// +build !linux
+
 package termstatus
 
 // IsProcessBackground reports whether the current process is running in the

internal/ui/termstatus/background_linux.go

@@ -0,0 +1,27 @@
+package termstatus
+
+import (
+	"github.com/restic/restic/internal/debug"
+
+	"golang.org/x/sys/unix"
+)
+
+// IsProcessBackground reports whether the current process is running in the
+// background. fd must be a file descriptor for the terminal.
+func IsProcessBackground(fd uintptr) bool {
+	bg, err := isProcessBackground(fd)
+	if err != nil {
+		debug.Log("Can't check if we are in the background. Using default behaviour. Error: %s\n", err.Error())
+		return false
+	}
+	return bg
+}
+
+func isProcessBackground(fd uintptr) (bool, error) {
+	// We need to use IoctlGetUint32 here, because pid_t is 32-bit even on
+	// 64-bit Linux. IoctlGetInt doesn't work on big-endian platforms:
+	// https://github.com/golang/go/issues/45585
+	// https://github.com/golang/go/issues/60429
+	pid, err := unix.IoctlGetUint32(int(fd), unix.TIOCGPGRP)
+	return int(pid) != unix.Getpgrp(), err
+}

internal/ui/termstatus/background_linux_test.go

@@ -1,5 +1,3 @@
-//go:build unix
-
 package termstatus
 
 import (
@@ -15,7 +13,7 @@ func TestIsProcessBackground(t *testing.T) {
 		t.Skipf("can't open terminal: %v", err)
 	}
 
-	_, err = isProcessBackground(int(tty.Fd()))
+	_, err = isProcessBackground(tty.Fd())
 	rtest.OK(t, err)
 
 	_ = tty.Close()

internal/ui/termstatus/background_unix.go

@@ -1,24 +0,0 @@
-//go:build unix
-
-package termstatus
-
-import "github.com/restic/restic/internal/debug"
-
-// IsProcessBackground reports whether the current process is running in the
-// background. fd must be a file descriptor for the terminal.
-func IsProcessBackground(fd uintptr) bool {
-	bg, err := isProcessBackground(int(fd))
-	if err != nil {
-		debug.Log("Can't check if we are in the background. Using default behaviour. Error: %s\n", err.Error())
-		return false
-	}
-	return bg
-}
-
-func isProcessBackground(fd int) (bg bool, err error) {
-	pgid, err := Tcgetpgrp(fd)
-	if err != nil {
-		return false, err
-	}
-	return pgid != Getpgrp(), nil
-}

internal/ui/termstatus/getpgrp_solaris.go

@@ -1,8 +0,0 @@
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Getpgrp() int {
-	pid, _ := unix.Getpgrp()
-	return pid
-}

internal/ui/termstatus/getpgrp_unix.go

@@ -1,7 +0,0 @@
-//go:build unix && !solaris
-
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Getpgrp() int { return unix.Getpgrp() }

internal/ui/termstatus/tcgetpgrp_linux.go

@@ -1,12 +0,0 @@
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Tcgetpgrp(ttyfd int) (int, error) {
-	// We need to use IoctlGetUint32 here, because pid_t is 32-bit even on
-	// 64-bit Linux. IoctlGetInt doesn't work on big-endian platforms:
-	// https://github.com/golang/go/issues/45585
-	// https://github.com/golang/go/issues/60429
-	pid, err := unix.IoctlGetUint32(ttyfd, unix.TIOCGPGRP)
-	return int(pid), err
-}

internal/ui/termstatus/tcgetpgrp_unix.go

@@ -1,9 +0,0 @@
-//go:build unix && !linux
-
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Tcgetpgrp(ttyfd int) (int, error) {
-	return unix.IoctlGetInt(ttyfd, unix.TIOCGPGRP)
-}

internal/ui/termstatus/tcsetpgrp_aix.go

@@ -1,10 +0,0 @@
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Tcsetpgrp(fd int, pid int) error {
-	// The second argument to IoctlSetPointerInt has type int on AIX,
-	// but the constant overflows 64-bit int, hence the two-step cast.
-	req := uint(unix.TIOCSPGRP)
-	return unix.IoctlSetPointerInt(fd, int(req), pid)
-}

internal/ui/termstatus/tcsetpgrp_unix.go

@@ -1,9 +0,0 @@
-//go:build unix && !aix
-
-package termstatus
-
-import "golang.org/x/sys/unix"
-
-func Tcsetpgrp(fd int, pid int) error {
-	return unix.IoctlSetPointerInt(fd, unix.TIOCSPGRP, pid)
-}