mirror of
https://github.com/restic/restic.git
synced 2026-05-21 01:25:24 +00:00
Merge pull request #21809 from MichaelEischer/doc-typos
doc: fix typos, unify section heading style and minor additions
This commit is contained in:
Michael Eischer
GitHub
@@ -14,11 +14,11 @@
|
||||
Introduction
|
||||
############
|
||||
|
||||
Restic is a fast and secure backup program. In the following sections, we will
|
||||
present typical workflows, starting with installing, preparing a new
|
||||
Restic is a fast and secure backup program. The following sections present
|
||||
typical workflows, starting with installing, preparing a new
|
||||
repository, and making the first backup.
|
||||
|
||||
Quickstart Guide
|
||||
Quickstart guide
|
||||
****************
|
||||
|
||||
To get started with a local repository, first define some environment variables:
|
||||
|
||||
+12
-12
@@ -45,8 +45,8 @@ package from the official community repos, e.g. using ``apk``:
|
||||
Arch Linux
|
||||
==========
|
||||
|
||||
On `Arch Linux <https://archlinux.org/>`__, there is a package called ``restic``
|
||||
installed from the official community repos, e.g. with ``pacman -S``:
|
||||
On `Arch Linux <https://archlinux.org/>`__, you can install the ``restic``
|
||||
package from the official community repos, e.g. with ``pacman -S``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -184,7 +184,7 @@ Windows
|
||||
|
||||
restic can be installed using either `Scoop <https://scoop.sh/>`__ or `WinGet <https://learn.microsoft.com/en-us/windows/package-manager/>`__.
|
||||
|
||||
Regardless of the method, the ``restic.exe`` binary will be added to your ``PATH`` automatically, making the ``restic`` command accessible in Powershell or CMD.
|
||||
Regardless of the method, the ``restic.exe`` binary will be added to your ``PATH`` automatically, making the ``restic`` command accessible in PowerShell or CMD.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -198,17 +198,17 @@ By default, WinGet will install restic into the ``User`` scope, which is typical
|
||||
|
||||
.. _official_binaries:
|
||||
|
||||
Official Binaries
|
||||
Official binaries
|
||||
*****************
|
||||
|
||||
Stable Releases
|
||||
Stable releases
|
||||
===============
|
||||
|
||||
You can download the latest stable release versions of restic from the `restic
|
||||
release page <https://github.com/restic/restic/releases/latest>`__. These builds
|
||||
are considered stable and releases are made regularly in a controlled manner.
|
||||
|
||||
There's both pre-compiled binaries for different platforms as well as the source
|
||||
There are both pre-compiled binaries for different platforms as well as the source
|
||||
code available for download. Just download and run the one matching your system.
|
||||
|
||||
On your first installation, if you desire, you can verify the integrity of your
|
||||
@@ -255,7 +255,7 @@ GitHub to verify their authenticity. No external programs are necessary.
|
||||
If you want to save the downloaded restic binary into a different file, pass
|
||||
the file name via the option ``--output``.
|
||||
|
||||
Unstable Builds
|
||||
Unstable builds
|
||||
===============
|
||||
|
||||
Another option is to use the latest builds for the master branch, available on
|
||||
@@ -264,11 +264,11 @@ the `restic beta download site
|
||||
and ready to run, and a new version is built every time a push is made to the
|
||||
master branch.
|
||||
|
||||
Docker Container
|
||||
Docker container
|
||||
****************
|
||||
|
||||
We're maintaining a bare docker container with just a few files and the restic
|
||||
binary, you can get it with `docker pull` like this:
|
||||
A minimal Docker image with just a few files and the restic
|
||||
binary is available; you can get it with ``docker pull`` like this:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -281,7 +281,7 @@ The container is also available on the GitHub Container Registry:
|
||||
$ docker pull ghcr.io/restic/restic
|
||||
|
||||
Restic relies on the hostname for various operations. Make sure to set a static
|
||||
hostname using `--hostname` when creating a Docker container, otherwise Docker
|
||||
hostname using ``--hostname`` when creating a Docker container, otherwise Docker
|
||||
will assign a random hostname each time.
|
||||
|
||||
The container additionally honors traditional ``nice`` `(man page) <https://man7.org/linux/man-pages/man1/nice.1.html>`__ and ``ionice`` `(man page) <https://man7.org/linux/man-pages/man1/ionice.1.html#OPTIONS>`__ directives via the following environment variables.
|
||||
@@ -301,7 +301,7 @@ The following example runs restic such that other CPU and IO requests have highe
|
||||
*Remember* that this invocation is explicitly telling your CPU and IO scheduler to deprioritize restic. This typically will result in a longer runtime. For a system with heavy load, this can be drastically longer.
|
||||
|
||||
|
||||
From Source
|
||||
From source
|
||||
***********
|
||||
|
||||
restic is written in the Go programming language and you need at least
|
||||
|
||||
@@ -22,8 +22,8 @@ To access the repository, a password (also called a key) must be specified. A
|
||||
repository can hold multiple keys that can all be used to access the repository.
|
||||
|
||||
This chapter explains how to create ("init") such a repository. The repository
|
||||
can be stored locally, or on some remote server or service. We'll first cover
|
||||
using a local repository; the remaining sections of this chapter cover all the
|
||||
can be stored locally, or on some remote server or service. This chapter first
|
||||
covers using a local repository; the remaining sections cover all the
|
||||
other options. You can skip to the next chapter once you've read the relevant
|
||||
section here.
|
||||
|
||||
@@ -89,7 +89,7 @@ command and enter the same password twice:
|
||||
On Linux, storing the backup repository on a CIFS (SMB) share or backing up
|
||||
data from a CIFS share is not recommended due to compatibility issues in
|
||||
older Linux kernels. Either use another backend or set the environment
|
||||
variable `GODEBUG` to `asyncpreemptoff=1`. Refer to GitHub issue
|
||||
variable ``GODEBUG`` to ``asyncpreemptoff=1``. Refer to GitHub issue
|
||||
:issue:`2659` for further explanations.
|
||||
|
||||
SFTP
|
||||
@@ -240,7 +240,7 @@ Amazon S3
|
||||
|
||||
Restic can backup data to any Amazon S3 bucket. However, in this case,
|
||||
changing the URL scheme is not enough since Amazon uses special security
|
||||
credentials to sign HTTP requests. By consequence, you must first setup
|
||||
credentials to sign HTTP requests. Consequently, you must first setup
|
||||
the following environment variables with the credentials you obtained
|
||||
while creating the bucket.
|
||||
|
||||
@@ -311,7 +311,7 @@ this command.
|
||||
Please note that knowledge of your password is required to access
|
||||
the repository. Losing your password means that your data is irrecoverably lost.
|
||||
|
||||
S3-compatible Storage
|
||||
S3-compatible storage
|
||||
*********************
|
||||
|
||||
For an S3-compatible storage service that is not Amazon, you can specify the URL to the server
|
||||
@@ -772,7 +772,7 @@ interaction. If you use emulation environments like
|
||||
`Cygwin <https://www.cygwin.com/>`__, which use terminals like
|
||||
``Mintty`` or ``rxvt``, you may get a password error.
|
||||
|
||||
You can workaround this by using a special tool called ``winpty`` (look
|
||||
You can work around this by using a special tool called ``winpty`` (look
|
||||
`here <https://www.msys2.org/wiki/Porting/>`__ and
|
||||
`here <https://github.com/rprichard/winpty>`__ for detailed information).
|
||||
On MSYS2, you can install ``winpty`` as follows:
|
||||
@@ -819,7 +819,7 @@ should belong to the appropriate group.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ restic backup -r sftp:restic@repohost:/srv/restic-repo
|
||||
$ restic -r sftp:restic@repohost:/srv/restic-repo backup
|
||||
|
||||
In the example, the command could be run by the local user ``root`` who can read
|
||||
all the files on the client host, and send them for backup using a remote user ``restic``
|
||||
|
||||
+19
-18
@@ -14,7 +14,7 @@
|
||||
Backing up
|
||||
##########
|
||||
|
||||
Now we're ready to backup some data. The contents of a directory at a
|
||||
You can now back up some data. The contents of a directory at a
|
||||
specific point in time is called a "snapshot" in restic. Run the
|
||||
following command and enter the repository password you chose above
|
||||
again:
|
||||
@@ -41,9 +41,9 @@ As you can see, restic created a backup of the directory and was pretty
|
||||
fast! The specific snapshot just created is identified by a sequence of
|
||||
hexadecimal characters, ``40dc1520`` in this case.
|
||||
|
||||
You can see that restic tells us it processed 1.720 GiB of data, this is the
|
||||
You can see that restic processed 1.720 GiB of data; this is the
|
||||
size of the files and directories in ``~/work`` on the local file system. It
|
||||
also tells us that only 1.200 GiB was added to the repository. This means that
|
||||
also reports that only 1.200 GiB was added to the repository. This means that
|
||||
some of the data was duplicate and restic was able to efficiently reduce it.
|
||||
The data compression also managed to compress the data down to 1.103 GiB.
|
||||
|
||||
@@ -64,7 +64,7 @@ estimated time of completion. The file paths displayed below the progress bar
|
||||
are the files currently being read by restic.
|
||||
|
||||
Be aware that the live status shows the processed files and not the transferred
|
||||
data. Transferred volume might be lower (due to de-duplication) or higher.
|
||||
data. Transferred volume might be lower (due to deduplication) or higher.
|
||||
|
||||
On Windows, the ``--use-fs-snapshot`` option will use Windows' Volume Shadow Copy
|
||||
Service (VSS) when creating backups. Restic will transparently create a VSS
|
||||
@@ -113,12 +113,12 @@ configured in the Windows registry under the following key:
|
||||
|
||||
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\BackupRestore\FilesNotToSnapshot
|
||||
|
||||
For more details refer the official Windows documentation e.g. the article
|
||||
For more details refer to the official Windows documentation e.g. the article
|
||||
``Registry Keys and Values for Backup and Restore``.
|
||||
|
||||
If you run the backup command again, restic will create another snapshot of
|
||||
your data, but this time it's even faster and no new data was added to the
|
||||
repository (since all data is already there). This is de-duplication at work!
|
||||
repository (since all data is already there). This is deduplication at work!
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -177,7 +177,7 @@ restic encounters:
|
||||
snapshot 8dc503fc saved
|
||||
|
||||
In fact several hosts may use the same repository to backup directories
|
||||
and files leading to a greater de-duplication.
|
||||
and files leading to a greater deduplication.
|
||||
|
||||
Now is a good time to run ``restic check`` to verify that all data
|
||||
is properly stored in the repository. You should run this command regularly
|
||||
@@ -319,7 +319,7 @@ Note that the snapshot metadata will always contain the absolute path.
|
||||
will not be able to detect unmodified files. This is because the change detection depends
|
||||
on the file path inside the snapshot.
|
||||
|
||||
Dry Runs
|
||||
Dry runs
|
||||
********
|
||||
|
||||
You can perform a backup in dry run mode to see what would happen without
|
||||
@@ -338,7 +338,7 @@ Combined with ``--verbose``, you can see a list of changes:
|
||||
|
||||
.. _backup-excluding-files:
|
||||
|
||||
Excluding Files
|
||||
Excluding files
|
||||
***************
|
||||
|
||||
You can exclude folders and files by specifying exclude patterns, currently
|
||||
@@ -348,14 +348,14 @@ the exclude options are:
|
||||
- ``--iexclude`` Same as ``--exclude`` but ignores the case of paths
|
||||
- ``--exclude-caches`` Specify once to exclude a folder's content if it contains `the special CACHEDIR.TAG file <https://bford.info/cachedir/>`__, but keep ``CACHEDIR.TAG``.
|
||||
- ``--exclude-file`` Specify one or more times to exclude items listed in a given file
|
||||
- ``--iexclude-file`` Same as ``exclude-file`` but ignores cases like in ``--iexclude``
|
||||
- ``--iexclude-file`` Same as ``--exclude-file`` but ignores cases like in ``--iexclude``
|
||||
- ``--exclude-if-present foo`` Specify one or more times to exclude a folder's content if it contains a file called ``foo`` (optionally having a given header, no wildcards for the file name supported)
|
||||
- ``--exclude-larger-than size`` Specify once to exclude files larger than the given size
|
||||
- ``--exclude-cloud-files`` Specify once to exclude online-only cloud files (such as OneDrive Files On-Demand, iCloud drive), currently only supported on Windows and macOS
|
||||
|
||||
Please see ``restic help backup`` for more specific information about each exclude option.
|
||||
|
||||
Let's say we have a file called ``excludes.txt`` with the following content:
|
||||
Suppose you have a file called ``excludes.txt`` with the following content:
|
||||
|
||||
::
|
||||
|
||||
@@ -402,8 +402,9 @@ This means, ``/bin`` matches ``/bin/bash`` but does not match ``/usr/bin/restic`
|
||||
Regular wildcards cannot be used to match over the directory separator ``/``,
|
||||
e.g. ``b*ash`` matches ``/bin/bash`` but does not match ``/bin/ash``. To match
|
||||
across an arbitrary number of subdirectories, use the special ``**`` wildcard.
|
||||
The ``**`` must be positioned between path separators. The pattern
|
||||
``foo/**/bar`` matches:
|
||||
|
||||
The ``**`` must be a **complete path component** (e.g. ``foo/**/bar``, not ``foo**`` or ``foo/**bar``).
|
||||
The pattern ``foo/**/bar`` matches:
|
||||
|
||||
* ``/dir1/foo/dir2/bar/file``
|
||||
* ``/foo/bar/file``
|
||||
@@ -508,7 +509,7 @@ include other filesystems like ``/sys`` and ``/proc``.
|
||||
.. note:: ``--one-file-system`` is currently unsupported on Windows, and will
|
||||
cause the backup to immediately fail with an error.
|
||||
|
||||
Files larger than a given size can be excluded using the `--exclude-larger-than`
|
||||
Files larger than a given size can be excluded using the ``--exclude-larger-than``
|
||||
option:
|
||||
|
||||
.. code-block:: console
|
||||
@@ -523,7 +524,7 @@ suffix the size value with one of ``k``/``K`` for KiB (1024 bytes), ``m``/``M``
|
||||
``g``/``G`` for GiB (1024^3 bytes) and ``t``/``T`` for TiB (1024^4 bytes), e.g. ``1k``, ``10K``, ``20m``,
|
||||
``20M``, ``30g``, ``30G``, ``2t`` or ``2T``).
|
||||
|
||||
Including Files
|
||||
Including files
|
||||
***************
|
||||
|
||||
The options ``--files-from``, ``--files-from-verbatim`` and ``--files-from-raw``
|
||||
@@ -553,7 +554,7 @@ The ``--files-from-raw`` option is a variant of ``--files-from-verbatim`` that
|
||||
requires each line in the file to be terminated by an ASCII NUL character (the
|
||||
``\0`` zero byte) instead of a newline, so that it can even handle file paths
|
||||
containing newlines in their name or are not encoded as UTF-8 (except on
|
||||
Windows, where the listed filenames must still be encoded in UTF-8. This option
|
||||
Windows, where the listed filenames must still be encoded in UTF-8). This option
|
||||
is the safest choice when generating the list of filenames from a script (e.g.
|
||||
GNU ``find`` with the ``-print0`` flag).
|
||||
|
||||
@@ -583,7 +584,7 @@ You can combine all three options with each other and with the normal file argum
|
||||
$ restic backup --files-from /tmp/files_to_backup /tmp/some_additional_file
|
||||
$ restic backup --files-from /tmp/glob-pattern --files-from-raw /tmp/generated-list /tmp/some_additional_file
|
||||
|
||||
Comparing Snapshots
|
||||
Comparing snapshots
|
||||
*******************
|
||||
|
||||
Restic has a ``diff`` command which shows the difference between two snapshots
|
||||
@@ -798,7 +799,7 @@ snapshot that then contains all but the unreadable files.
|
||||
For use of these exit status codes in scripts and other automation tools, see :ref:`exit-codes`.
|
||||
To manually inspect the exit code in e.g. Linux, run ``echo $?``.
|
||||
|
||||
Environment Variables
|
||||
Environment variables
|
||||
*********************
|
||||
|
||||
In addition to command-line options, restic supports passing various options in
|
||||
|
||||
@@ -60,7 +60,7 @@ Or filter by host:
|
||||
|
||||
Combining filters is also possible.
|
||||
|
||||
Furthermore you can group the output by the same filters (host, paths, tags):
|
||||
Furthermore, you can group the output by the same filters (host, paths, tags):
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -224,6 +224,11 @@ example from a local to a remote repository, you can use the ``copy`` command:
|
||||
|
||||
The example command copies all snapshots from the source repository
|
||||
``/srv/restic-repo`` to the destination repository ``/srv/restic-repo-copy``.
|
||||
The destination repository must already exist; when creating it for the first
|
||||
time, use ``init --from-repo`` with ``--copy-chunker-params`` as described in
|
||||
:ref:`copy-deduplication` below so that deduplication works across both
|
||||
repositories.
|
||||
|
||||
Snapshots which have previously been copied between repositories will
|
||||
be skipped by later copy runs. Information about skipped snapshots is only
|
||||
printed when ``--verbose`` is passed to the command. For efficiency reasons,
|
||||
@@ -248,7 +253,7 @@ can also be set as environment variables ``$RESTIC_FROM_REPOSITORY`` or
|
||||
the password can be read from a file ``--from-password-file`` or from a command
|
||||
``--from-password-command``.
|
||||
Alternatively the environment variables ``$RESTIC_FROM_PASSWORD_COMMAND`` and
|
||||
``$RESTIC_FROM_PASSWORD_FILE`` can be used. It is also possible to directly
|
||||
``$RESTIC_FROM_PASSWORD_FILE`` can be used. You can also directly
|
||||
pass the password via ``$RESTIC_FROM_PASSWORD``. The key which should be used
|
||||
for decryption can be selected by passing its ID via the flag ``--from-key-hint``
|
||||
or the environment variable ``$RESTIC_FROM_KEY_HINT``.
|
||||
@@ -260,7 +265,7 @@ or the environment variable ``$RESTIC_FROM_KEY_HINT``.
|
||||
repository. You can avoid this limitation by using the rclone backend
|
||||
along with remotes which are configured in rclone.
|
||||
|
||||
.. note:: If `copy` is aborted, `copy` will resume the interrupted copying when it is run again. It's possible that up to 10 minutes of progress can be lost because the repository index is only updated from time to time.
|
||||
.. note:: If ``copy`` is aborted, ``copy`` will resume the interrupted copying when it is run again. It's possible that up to 10 minutes of progress can be lost because the repository index is only updated from time to time.
|
||||
|
||||
.. _copy-filtering-snapshots:
|
||||
|
||||
@@ -274,13 +279,15 @@ and/or a comma-separated tag list:
|
||||
|
||||
$ restic -r /srv/restic-repo-copy copy --from-repo /srv/restic-repo --host luigi --path /srv --tag foo,bar
|
||||
|
||||
It is also possible to explicitly specify the list of snapshots to copy, in
|
||||
You can also explicitly specify the list of snapshots to copy, in
|
||||
which case only these instead of all snapshots will be copied:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ restic -r /srv/restic-repo-copy copy --from-repo /srv/restic-repo 410b18a2 4e5d5487 latest
|
||||
|
||||
.. _copy-deduplication:
|
||||
|
||||
Ensuring deduplication for copied snapshots
|
||||
-------------------------------------------
|
||||
|
||||
@@ -293,7 +300,7 @@ identical chunks and therefore deduplication also works for snapshots copied bet
|
||||
these repositories.
|
||||
|
||||
The chunker parameters are generated once when creating a new (destination) repository.
|
||||
That is for a copy destination repository we have to instruct restic to initialize it
|
||||
For a copy destination repository, you must instruct restic to initialize it
|
||||
using the same chunker parameters as the source repository:
|
||||
|
||||
.. code-block:: console
|
||||
@@ -308,7 +315,7 @@ Removing files from snapshots
|
||||
|
||||
Snapshots sometimes turn out to include more files than intended. Instead of
|
||||
removing the snapshots entirely and running the corresponding backup commands
|
||||
again (which is not always practical after the fact) it is possible to remove
|
||||
again (which is not always practical after the fact), you can remove
|
||||
the unwanted files from affected snapshots by rewriting them using the
|
||||
``rewrite`` command:
|
||||
|
||||
@@ -345,7 +352,7 @@ to those files that you are really interested in. An example could be all pictur
|
||||
from a snapshot:
|
||||
``restic rewrite -r ... --iinclude "*.jpg" --iinclude "*.jpeg" --iinclude "*.png"``.
|
||||
|
||||
It is possible to rewrite only a subset of snapshots by filtering them the same
|
||||
You can rewrite only a subset of snapshots by filtering them the same
|
||||
way as for the ``copy`` command, see :ref:`copy-filtering-snapshots`.
|
||||
|
||||
The option ``--snapshot-summary`` can be used to attach summary data to existing
|
||||
@@ -374,7 +381,7 @@ perform.
|
||||
version of restic or some third-party implementation.
|
||||
|
||||
To convert a snapshot into the format expected by the ``rewrite`` command
|
||||
use ``restic repair snapshots <snapshotID>``.
|
||||
use ``restic -r /srv/restic-repo repair snapshots <snapshotID>``.
|
||||
|
||||
Modifying metadata of snapshots
|
||||
===============================
|
||||
@@ -385,7 +392,7 @@ This is possible using the ``rewrite`` command with the option ``--new-host`` fo
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ restic rewrite --new-host newhost --new-time "1999-01-01 11:11:11"
|
||||
$ restic -r /srv/restic-repo rewrite --new-host newhost --new-time "1999-01-01 11:11:11"
|
||||
|
||||
repository b7dbade3 opened (version 2, compression level auto)
|
||||
[0:00] 100.00% 1 / 1 index files loaded
|
||||
@@ -440,7 +447,7 @@ detect this and yield the same error as when you tried to restore:
|
||||
load indexes
|
||||
error: error loading index de30f3231ca2e6a59af4aa84216dfe2ef7339c549dc11b09b84000997b139628: LoadRaw(<index/de30f3231c>): invalid data returned
|
||||
|
||||
The repository index is damaged and must be repaired. You must run `restic repair index' to correct this.
|
||||
The repository index is damaged and must be repaired. You must run ``restic repair index`` to correct this.
|
||||
|
||||
Fatal: repository contains errors
|
||||
|
||||
@@ -550,10 +557,10 @@ Finding things in the repository
|
||||
The ``restic find`` command searches for files or directories stored
|
||||
in the repository.
|
||||
|
||||
find files and directories
|
||||
--------------------------
|
||||
Finding files and directories
|
||||
-----------------------------
|
||||
|
||||
If you want to find files or directories in the repository, you either specific filename(s)
|
||||
If you want to find files or directories in the repository, you either specify filename(s)
|
||||
or a pattern which represents filename(s).
|
||||
The use of file patterns is described in :ref:`backup-excluding-files`.
|
||||
Here is an example:
|
||||
@@ -580,8 +587,8 @@ The following example searches for files which have a modification date in the y
|
||||
All these commands work in ``--json`` mode as well, for output details for the
|
||||
various options please refer to :ref:`find`.
|
||||
|
||||
find blobs, trees or packfiles
|
||||
------------------------------
|
||||
Finding blobs, trees, or packfiles
|
||||
----------------------------------
|
||||
|
||||
The other options of the ``find`` command are devoted to finding blobs, trees and packfiles.
|
||||
These are typically not used by the normal user, but can help debugging a problem
|
||||
@@ -612,7 +619,7 @@ Upgrading to repository version 2 is a two step process: first run
|
||||
then upgrade the repository version. Repository problems must be corrected
|
||||
before the migration will be possible. After the migration is complete, run
|
||||
``prune`` to compress the repository metadata. To limit the amount of data
|
||||
rewritten in at once, you can use the ``prune --max-repack-size size``
|
||||
rewritten at once, you can use the ``prune --max-repack-size size``
|
||||
parameter, see :ref:`customize-pruning` for more details.
|
||||
|
||||
File contents stored in the repository will not be rewritten, data from new
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
" for paragraphs
|
||||
|
||||
#################
|
||||
Tuning Parameters
|
||||
Tuning parameters
|
||||
#################
|
||||
|
||||
Restic offers a few parameters that allow tuning the backup and other operations.
|
||||
@@ -19,7 +19,7 @@ benefit from different non-default values. As the restic commands evolve over
|
||||
time, the optimal value for each parameter can also change across restic versions.
|
||||
|
||||
|
||||
Disabling Backup Progress Estimation
|
||||
Disabling backup progress estimation
|
||||
====================================
|
||||
|
||||
When you start a backup, restic will concurrently count the number of files and
|
||||
@@ -29,7 +29,7 @@ FUSE mounts. To avoid this overhead at the cost of not seeing a progress
|
||||
estimate, use the ``--no-scan`` option of the ``backup`` command which disables
|
||||
this file scanning.
|
||||
|
||||
Backend Connections
|
||||
Backend connections
|
||||
===================
|
||||
|
||||
Restic uses a global limit for the number of concurrent connections to a backend.
|
||||
@@ -44,12 +44,12 @@ upload times for single temporary packs, which can lead to more disk wear on SSD
|
||||
:ref:`pack_size`).
|
||||
|
||||
|
||||
CPU Usage
|
||||
CPU usage
|
||||
=========
|
||||
|
||||
By default, restic uses all available CPU cores. You can set the environment variable
|
||||
`GOMAXPROCS` to limit the number of used CPU cores. For example to use a single CPU core,
|
||||
use `GOMAXPROCS=1`. Limiting the number of usable CPU cores, can slightly reduce the memory
|
||||
``GOMAXPROCS`` to limit the number of used CPU cores. For example to use a single CPU core,
|
||||
use ``GOMAXPROCS=1``. Limiting the number of usable CPU cores can slightly reduce the memory
|
||||
usage of restic.
|
||||
|
||||
|
||||
@@ -63,7 +63,7 @@ and storage space. This setting is only applied for the single run of restic, bu
|
||||
set via the environment variable ``RESTIC_COMPRESSION``.
|
||||
|
||||
|
||||
Data Verification
|
||||
Data verification
|
||||
=================
|
||||
|
||||
To prevent the upload of corrupted data to the repository, which can happen due
|
||||
@@ -77,7 +77,7 @@ Otherwise, data corruption due to hardware issues or software bugs might go
|
||||
unnoticed.
|
||||
|
||||
|
||||
File Read Concurrency
|
||||
File read concurrency
|
||||
=====================
|
||||
|
||||
When backing up files from fast storage like NVMe disks, it can be beneficial to increase
|
||||
@@ -89,7 +89,7 @@ the ``backup`` command.
|
||||
|
||||
.. _pack_size:
|
||||
|
||||
Pack Size
|
||||
Pack size
|
||||
=========
|
||||
|
||||
In certain instances, such as very large repositories (in the TiB range) or very fast
|
||||
@@ -120,7 +120,7 @@ increases the chance of these files being written to disk. This can increase dis
|
||||
for SSDs.
|
||||
|
||||
|
||||
Feature Flags
|
||||
Feature flags
|
||||
=============
|
||||
|
||||
Feature flags allow disabling or enabling certain experimental restic features. The flags
|
||||
|
||||
+12
-12
@@ -134,8 +134,8 @@ values are supported:
|
||||
newer modification time (mtime).
|
||||
* ``--overwrite never``: never overwrite existing files.
|
||||
|
||||
Delete files not in snapshot
|
||||
----------------------------
|
||||
Deleting files not in snapshot
|
||||
------------------------------
|
||||
|
||||
When restoring into a directory that already contains files, it can be useful to remove all
|
||||
files that do not exist in the snapshot. For this, pass the ``--delete`` option to the ``restore``
|
||||
@@ -152,15 +152,15 @@ options will be deleted. For example, the command
|
||||
``restic -r /srv/restic-repo restore 79766175:/home/user/work --target /tmp/restore --include /foo --delete``
|
||||
would only delete files within ``/tmp/restore/foo``.
|
||||
|
||||
When using ``--target / --delete`` then the ``restore`` command only works if either an ``--include``
|
||||
or ``--exclude`` option is also specified. This ensures that one cannot accidentally delete
|
||||
When using ``--target`` and ``--delete`` then the ``restore`` command only works if either an ``--include``
|
||||
or ``--exclude`` option is also specified. This ensures that you cannot accidentally delete
|
||||
the whole system.
|
||||
|
||||
The ``--delete`` option also allows overwriting a non-empty directory if the snapshot contains a
|
||||
file with the same name.
|
||||
|
||||
Dry run
|
||||
-------
|
||||
Dry runs
|
||||
--------
|
||||
|
||||
As restore operations can take a long time, it can be useful to perform a dry-run to
|
||||
see what would be restored without having to run the full restore operation. The
|
||||
@@ -186,8 +186,8 @@ To reliably determine which files would be updated, a dry-run also verifies the
|
||||
already existing files according to the specified overwrite behavior. To skip these checks
|
||||
either specify ``--overwrite never`` or specify a non-existing ``--target`` directory.
|
||||
|
||||
Restore using mount
|
||||
===================
|
||||
Restoring using mount
|
||||
=====================
|
||||
|
||||
Browsing your backup as a regular file system is also very easy. First,
|
||||
create a mount point such as ``/mnt/restic`` and then use the following
|
||||
@@ -223,7 +223,7 @@ Printing files to stdout
|
||||
========================
|
||||
|
||||
Sometimes it's helpful to print files to stdout so that other programs can read
|
||||
the data directly. This can be achieved by using the `dump` command, like this:
|
||||
the data directly. This can be achieved by using the ``dump`` command, like this:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -247,7 +247,7 @@ snapshots in a repository:
|
||||
----------------------------------------------------------------------
|
||||
|
||||
Here, restic would resolve ``latest`` to the snapshot ``1541acae``, which does
|
||||
not contain the file we'd like to print at all (``production.sql``). In this
|
||||
not contain the file you want to print at all (``production.sql``). In this
|
||||
case, you can pass restic the snapshot ID of the snapshot you like to restore:
|
||||
|
||||
.. code-block:: console
|
||||
@@ -273,7 +273,7 @@ See :ref:`absolute-and-relative-paths` for details. Use ``ls`` to determine the
|
||||
/other
|
||||
/other/work
|
||||
|
||||
It is also possible to ``dump`` the contents of a whole folder structure to
|
||||
You can also ``dump`` the contents of a whole folder structure to
|
||||
stdout. To retain the information about the files and folders restic will
|
||||
output the contents in the tar (default) or zip format:
|
||||
|
||||
@@ -292,7 +292,7 @@ To include the folder content at the root of the archive, you can use the ``<sna
|
||||
|
||||
$ restic -r /srv/restic-repo dump latest:/home/other/work / > restore.tar
|
||||
|
||||
It is also possible to ``dump`` the contents of a selected snapshot and folder
|
||||
You can also ``dump`` the contents of a selected snapshot and folder
|
||||
structure to a file using the ``--target`` flag.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
+17
-15
@@ -32,8 +32,8 @@ It is advisable to run ``restic check`` after pruning, to make sure
|
||||
you are alerted, should the internal data structures of the repository
|
||||
be damaged.
|
||||
|
||||
Remove a single snapshot
|
||||
************************
|
||||
Removing a single snapshot
|
||||
**************************
|
||||
|
||||
The command ``snapshots`` can be used to list all snapshots in a
|
||||
repository like this:
|
||||
@@ -234,9 +234,11 @@ command to the same value.
|
||||
|
||||
Additionally, you can restrict the policy to only process snapshots which have a
|
||||
particular hostname with the ``--host`` parameter, or tags with the ``--tag``
|
||||
option. When multiple tags are specified, only the snapshots which have all the
|
||||
tags are considered. For example, the following command removes all but the
|
||||
latest snapshot of all snapshots that have the tag ``foo``:
|
||||
option. Each ``--tag`` argument defines a tag list: tags separated by commas
|
||||
mean the snapshot must have all of those tags (AND within the list). When
|
||||
``--tag`` is given multiple times, the lists are combined with OR (the snapshot
|
||||
is considered if it matches any list). For example, the following command
|
||||
removes all but the latest snapshot of all snapshots that have the tag ``foo``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -278,11 +280,11 @@ might be spread over a longer period. If what you want is to keep daily
|
||||
snapshots for the last week, weekly for the last month, monthly for the last
|
||||
year and yearly for the last 75 years, you can instead specify ``forget
|
||||
--keep-within-daily 7d --keep-within-weekly 1m --keep-within-monthly 1y
|
||||
--keep-within-yearly 75y`` (note that `1w` is not a recognized duration, so
|
||||
you will have to specify `7d` instead).
|
||||
--keep-within-yearly 75y`` (note that ``1w`` is not a recognized duration, so
|
||||
you will have to specify ``7d`` instead).
|
||||
|
||||
The processed snapshots are evaluated against all ``--keep-*`` options but a
|
||||
snapshot only need to match a single option to be kept (the results are ORed).
|
||||
snapshot only needs to match a single option to be kept (the results are ORed).
|
||||
This means that the most recent snapshot would match both hourly,
|
||||
daily and weekly ``--keep-*`` options, and possibly more depending on calendar.
|
||||
|
||||
@@ -393,7 +395,7 @@ removes all snapshots with tag ``example``.
|
||||
Security considerations in append-only mode
|
||||
===========================================
|
||||
|
||||
.. note:: TL;DR: With append-only repositories, one should specifically use the
|
||||
.. note:: TL;DR: With append-only repositories, you should specifically use the
|
||||
``--keep-within`` option of the ``forget`` command when removing snapshots.
|
||||
|
||||
To prevent a compromised backup client from deleting its backups (for example
|
||||
@@ -402,7 +404,7 @@ repository in a so-called append-only mode. This means that the repository is
|
||||
served in such a way that it can only be written to and read from, while delete
|
||||
and overwrite operations are denied. Restic's `rest-server`_ features an
|
||||
append-only mode, but few other standard backends do. To support append-only
|
||||
with such backends, one can use `rclone`_ as a complement in between the backup
|
||||
with such backends, you can use `rclone`_ as a complement in between the backup
|
||||
client and the backend service.
|
||||
|
||||
.. _rest-server: https://github.com/restic/rest-server/
|
||||
@@ -444,10 +446,10 @@ all legitimate snapshots.
|
||||
|
||||
.. _customize-pruning:
|
||||
|
||||
Customize pruning
|
||||
*****************
|
||||
Customizing pruning
|
||||
*******************
|
||||
|
||||
To understand the custom options, we first explain how the pruning process works:
|
||||
To understand the custom options, this section first explains how the pruning process works:
|
||||
|
||||
1. All snapshots and directories within snapshots are scanned to determine
|
||||
which data is still in use.
|
||||
@@ -526,8 +528,8 @@ is available as a method of last resort. It allows prune to work with little to
|
||||
space. However, a **failed** ``prune`` run can cause the repository to become
|
||||
**temporarily unusable**. Therefore, make sure that you have a stable connection to the
|
||||
repository storage, before running this command. In case the command fails, it may become
|
||||
necessary to manually remove all files from the `index/` folder of the repository and
|
||||
run `repair index` afterwards.
|
||||
necessary to manually remove all files from the ``index/`` folder of the repository and
|
||||
run ``restic repair index`` afterwards.
|
||||
|
||||
To prevent accidental usages of the ``--unsafe-recover-no-free-space`` option it is
|
||||
necessary to first run ``prune --unsafe-recover-no-free-space SOME-ID`` and then replace
|
||||
|
||||
@@ -21,9 +21,9 @@ it. So… I’m going to use restic for my personal backups.*" `Filippo Valsorda
|
||||
|
||||
.. _Filippo Valsorda: https://words.filippo.io/restic-cryptography/
|
||||
|
||||
**********************
|
||||
Manage repository keys
|
||||
**********************
|
||||
************************
|
||||
Managing repository keys
|
||||
************************
|
||||
|
||||
The ``key`` command allows you to set multiple access keys or passwords
|
||||
per repository. In fact, you can use the ``list``, ``add``, ``remove``, and
|
||||
|
||||
+17
-10
@@ -19,7 +19,7 @@ when you use restic via scripts.
|
||||
|
||||
.. _environment-variables:
|
||||
|
||||
Environment Variables
|
||||
Environment variables
|
||||
*********************
|
||||
|
||||
In addition to command-line options, restic supports passing various options in
|
||||
@@ -42,6 +42,13 @@ environment variables, which are listed below.
|
||||
RESTIC_PACK_SIZE Target size for pack files
|
||||
RESTIC_READ_CONCURRENCY Concurrency for file reads
|
||||
|
||||
RESTIC_FROM_REPOSITORY Source repository for copy (replaces --from-repo)
|
||||
RESTIC_FROM_REPOSITORY_FILE File containing source repository for copy (replaces --from-repository-file)
|
||||
RESTIC_FROM_PASSWORD Password for the source repository (copy)
|
||||
RESTIC_FROM_PASSWORD_FILE Password file for the source repository (replaces --from-password-file)
|
||||
RESTIC_FROM_PASSWORD_COMMAND Command to obtain source repository password (replaces --from-password-command)
|
||||
RESTIC_FROM_KEY_HINT Key ID to try first when opening the source repository (replaces --from-key-hint)
|
||||
|
||||
TMPDIR Location for temporary files (except Windows)
|
||||
TMP Location for temporary files (only Windows)
|
||||
|
||||
@@ -109,8 +116,8 @@ The external programs that restic may execute include ``rclone`` (for rclone
|
||||
backends) and ``ssh`` (for the SFTP backend). These may respond to further
|
||||
environment variables and configuration files; see their respective manuals.
|
||||
|
||||
Check if a repository is already initialized
|
||||
********************************************
|
||||
Checking if a repository is already initialized
|
||||
***********************************************
|
||||
|
||||
You may find a need to check if a repository is already initialized,
|
||||
perhaps to prevent your script from trying to initialize a repository multiple
|
||||
@@ -169,15 +176,15 @@ JSON output
|
||||
***********
|
||||
|
||||
Restic outputs JSON data to ``stdout`` if requested with the ``--json`` flag.
|
||||
The structure of that data varies depending on the circumstance. The
|
||||
JSON output of most restic commands are documented here.
|
||||
The structure of that data varies depending on the circumstance. The
|
||||
JSON output of most restic commands is documented here.
|
||||
|
||||
.. note::
|
||||
Not all commands support JSON output. If a command does not support JSON output,
|
||||
feel free to submit a pull request!
|
||||
|
||||
.. warning::
|
||||
We try to keep the JSON output backwards compatible. However, new message types
|
||||
The JSON output is intended to remain backwards compatible. However, new message types
|
||||
or fields may be added at any time. Similarly, enum-like fields for which a fixed
|
||||
list of allowed values is documented may be extended at any time.
|
||||
|
||||
@@ -279,7 +286,7 @@ These errors are printed on ``stderr``.
|
||||
| ``item`` | Usually, the path of the problematic file | string |
|
||||
+-------------------+-------------------------------------------+--------+
|
||||
|
||||
Verbose Status
|
||||
Verbose status
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
Verbose status provides details about the progress, including details about backed up files.
|
||||
@@ -353,7 +360,7 @@ cat
|
||||
---
|
||||
|
||||
The ``cat`` command returns data about various objects in the repository, which
|
||||
are stored in JSON form. Specifying ``--json`` or ``--quiet`` will suppress any
|
||||
are stored in JSON form. Specifying ``--json`` or ``--quiet`` will suppress any
|
||||
non-JSON messages the command generates.
|
||||
|
||||
|
||||
@@ -750,11 +757,11 @@ These errors are printed on ``stderr``.
|
||||
| ``item`` | Usually, the path of the problematic file | string |
|
||||
+-------------------+-------------------------------------------+--------+
|
||||
|
||||
Verbose Status
|
||||
Verbose status
|
||||
^^^^^^^^^^^^^^
|
||||
|
||||
Verbose status provides details about the progress, including details about restored files.
|
||||
Only printed if `--verbose=2` is specified.
|
||||
Only printed if ``--verbose=2`` is specified.
|
||||
|
||||
+------------------+--------------------------------------------------------+--------+
|
||||
| ``message_type`` | Always "verbose_status" | string |
|
||||
|
||||
+16
-16
@@ -41,8 +41,8 @@ bugfixes, and improvements to simplify the repair of a repository. It might also
|
||||
contain a fix for your repository problems!
|
||||
|
||||
|
||||
1. Find out what is damaged
|
||||
***************************
|
||||
1. Finding damaged data
|
||||
***********************
|
||||
|
||||
The first step is always to check the repository.
|
||||
|
||||
@@ -81,7 +81,7 @@ somewhere. Please include the check output and additional information that might
|
||||
help locate the problem.
|
||||
|
||||
If ``check`` detects damaged pack files, it will show instructions on how to repair
|
||||
them using the ``repair packs`` command. Use that command instead of the "Repair the
|
||||
them using the ``repair packs`` command. Use that command instead of the "Repairing the
|
||||
index" section in this guide.
|
||||
|
||||
If you are interested to check only specific snapshots, you can now
|
||||
@@ -91,8 +91,8 @@ will then be checked for consistency and read when either ``--read-data`` or
|
||||
``--read-data-subset`` is given.
|
||||
|
||||
|
||||
2. Backup the repository
|
||||
************************
|
||||
2. Backing up the repository
|
||||
****************************
|
||||
|
||||
Create a full copy of the repository if possible. Or at the very least make a
|
||||
copy of the ``index`` and ``snapshots`` folders. This will allow you to roll back
|
||||
@@ -115,12 +115,12 @@ whether your issue is already known and solved. Please take a look at the
|
||||
`forum`_ and `GitHub issues <https://github.com/restic/restic/issues>`_.
|
||||
|
||||
|
||||
3. Repair the index
|
||||
*******************
|
||||
3. Repairing the index
|
||||
**********************
|
||||
|
||||
.. note::
|
||||
|
||||
If the `check` command tells you to run `restic repair packs`, then use that
|
||||
If the ``check`` command tells you to run ``restic repair packs``, then use that
|
||||
command instead. It will repair the damaged pack files and also update the index.
|
||||
|
||||
Restic relies on its index to contain correct information about what data is
|
||||
@@ -149,8 +149,8 @@ Please note that it is not recommended to repair the index unless the repository
|
||||
is actually damaged.
|
||||
|
||||
|
||||
4. Run all backups (optional)
|
||||
*****************************
|
||||
4. Running all backups (optional)
|
||||
*********************************
|
||||
|
||||
With a correct index, the ``backup`` command guarantees that newly created
|
||||
snapshots can be restored successfully. It can also heal older snapshots,
|
||||
@@ -165,8 +165,8 @@ To check if the repository is fully repaired, you can run ``restic check``
|
||||
To get a list of still damaged files, you can run ``restic repair snapshots --dry-run``.
|
||||
Look for ``would save new snapshot`` messages to find affected snapshots.
|
||||
|
||||
5. Remove missing data from snapshots
|
||||
*************************************
|
||||
5. Removing missing data from snapshots
|
||||
***************************************
|
||||
|
||||
If your repository is still missing data, then you can use the ``repair snapshots``
|
||||
command to remove all inaccessible data from the snapshots. That is, this will
|
||||
@@ -191,11 +191,11 @@ modified snapshots using the ``forget`` command. In the example above, you'd hav
|
||||
to run ``restic forget 6979421e``.
|
||||
|
||||
|
||||
6. Check the repository again
|
||||
*****************************
|
||||
6. Checking the repository again
|
||||
********************************
|
||||
|
||||
Phew, we're almost done now. To make sure that the repository has been successfully
|
||||
repaired please run ``check`` again.
|
||||
As a final step, run ``check`` again to make sure that the repository has been successfully
|
||||
repaired.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
|
||||
+10
-11
@@ -207,8 +207,7 @@ bucket's name, region, and your user's API credentials.
|
||||
.. code-block:: console
|
||||
|
||||
$ unset HISTFILE
|
||||
$ export AWS_DEFAULT_REGION="eu-west-1"
|
||||
$ export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/restic-demo"
|
||||
$ export RESTIC_REPOSITORY="s3:s3.eu-west-1.amazonaws.com/restic-demo"
|
||||
$ export AWS_ACCESS_KEY_ID="AKIAJAJSLTZCAZ4SRI5Q"
|
||||
$ export AWS_SECRET_ACCESS_KEY="LaJtZPoVvGbXsaD2LsxvJZF/7LRi4FhT0TK4gDQq"
|
||||
$ export RESTIC_PASSWORD="I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5"
|
||||
@@ -221,7 +220,7 @@ repository:
|
||||
.. code-block:: console
|
||||
|
||||
$ restic init
|
||||
created restic backend b5c661a86a at s3:https://s3.amazonaws.com/restic-demo
|
||||
created restic backend b5c661a86a at s3:s3.eu-west-1.amazonaws.com/restic-demo
|
||||
|
||||
Please note that knowledge of your password is required to access
|
||||
the repository. Losing your password means that your data is
|
||||
@@ -292,13 +291,13 @@ least possible privileges.
|
||||
Fortunately, Linux has functionality to divide root's power into
|
||||
single separate *capabilities*. The *CAP_DAC_READ_SEARCH* capability
|
||||
allows the current process to "Bypass file read permission checks and
|
||||
directory read and execute permission checks", which is what we need to
|
||||
directory read and execute permission checks", which is what you need to
|
||||
back up a system.
|
||||
|
||||
Using ambient capabilities (recommended)
|
||||
========================================
|
||||
|
||||
First we create a new user called ``restic`` that is going to create
|
||||
First, create a new user called ``restic`` that is going to create
|
||||
the backups:
|
||||
|
||||
.. code-block:: console
|
||||
@@ -362,21 +361,21 @@ execution, the system will read the assigned capabilities and assign
|
||||
them to the process. This is less secure than using ambient capabilities
|
||||
as anyone who is able to execute the binary can make use of the capability.
|
||||
|
||||
First we create a new user called ``restic`` that is going to create
|
||||
First, create a new user called ``restic`` that is going to create
|
||||
the backups:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# useradd --system --create-home --shell /sbin/nologin restic
|
||||
|
||||
Then we copy the restic binary into the user's home directory:
|
||||
Then copy the restic binary into the user's home directory:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# mkdir /home/restic/bin
|
||||
# cp /usr/bin/restic /home/restic/bin/restic
|
||||
|
||||
Before we assign any special capability to the restic binary we
|
||||
Before assigning any special capability to the restic binary,
|
||||
restrict its permissions so that only root and the newly created
|
||||
restic user can execute it. Otherwise any user could use the
|
||||
privileged restic binary to access any file.
|
||||
@@ -386,7 +385,7 @@ privileged restic binary to access any file.
|
||||
# chown root:restic /home/restic/bin/restic
|
||||
# chmod 750 /home/restic/bin/restic
|
||||
|
||||
Finally we can use ``setcap`` to add an extended attribute to the
|
||||
Finally, use ``setcap`` to add an extended attribute to the
|
||||
restic binary. On every execution the system will read the extended
|
||||
attribute, interpret it and assign capabilities accordingly.
|
||||
|
||||
@@ -429,8 +428,8 @@ Running a local repository server
|
||||
=================================
|
||||
|
||||
On the internal host, download and run the latest `release <https://github.com/restic/rest-server/releases>`__
|
||||
of REST-server to act as the repository server. In this example we are using
|
||||
the ``--no-auth`` option to not require authentication when connecting to it:
|
||||
of REST-server to act as the repository server. In this example the
|
||||
``--no-auth`` option is used to not require authentication when connecting to it:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
|
||||
@@ -166,7 +166,7 @@ The classical helpers for integration tests are, amongst others:
|
||||
- ``testListSnapshots(t, env.gopts, <n>)``: check that there are <n> snapshots in the repository
|
||||
- ``testRunCheck(t, env.gopts)``: check that the repository is sound and happy
|
||||
- the above mentioned ``rtest.OK()``, ``rtest.Equals()``, ``rtest.Assert()`` helpers
|
||||
- ``withCaptureStdout()`` and ``withTermStatus()`` wrappers: both functions are found in ``cmd/restic/integration_helpers_test.go`` for creating an environment where one can analyze the output created by the ``testRunXXX()`` command, particularly when checking JSON output
|
||||
- ``withCaptureStdout()`` and ``withTermStatus()`` wrappers: both functions are found in ``cmd/restic/integration_helpers_test.go`` for creating an environment where you can analyze the output created by the ``testRunXXX()`` command, particularly when checking JSON output
|
||||
|
||||
Integration tests test the overall workings of a command. Integration tests are used for commands and
|
||||
are stored in the same directory ``cmd/restic``. The recommended naming convention is
|
||||
@@ -221,7 +221,7 @@ Example: this is a typical setup for a backup / find scenario
|
||||
// ``testRunFind()`` uses ``withCaptureStdout()`` to capture output text (in ``results``)
|
||||
results := testRunFind(t, false, FindOptions{}, env.gopts, "testfile")
|
||||
|
||||
// there is always a ``\n`` at the end of the output!
|
||||
// there is always a ``\n`` at the end of the output!
|
||||
lines := strings.Split(string(results), "\n")
|
||||
|
||||
// make sure that we have correct output
|
||||
|
||||
+2
-2
@@ -10,11 +10,11 @@ Each repository has its own cache sub-directory, consisting of the repository ID
|
||||
which is chosen at ``init``. All cache directories for different repositories are
|
||||
independent of each other.
|
||||
|
||||
Snapshots, Data and Indexes
|
||||
Snapshots, data and indexes
|
||||
===========================
|
||||
|
||||
Snapshot, Data and Index files are cached in the sub-directories ``snapshots``,
|
||||
``data`` and ``index``, as read from the repository.
|
||||
``data`` and ``index``, as read from the repository.
|
||||
|
||||
Expiry
|
||||
======
|
||||
|
||||
+13
-13
@@ -16,7 +16,7 @@ information like the SHA-256 hash of the data and its length.
|
||||
|
||||
*Snapshot*: A Snapshot stands for the state of a file or directory that
|
||||
has been backed up at some point in time. The state here means the
|
||||
content and meta data like the name and modification time for the file
|
||||
content and metadata like the name and modification time for the file
|
||||
or the directory and its contents.
|
||||
|
||||
*Storage ID*: A storage ID is the SHA-256 hash of the content stored in
|
||||
@@ -211,7 +211,7 @@ tree blobs may be compressed with the zstandard compression algorithm.
|
||||
|
||||
In repository format version 1, data and tree blobs should be stored in
|
||||
separate pack files. In version 2, they must be stored in separate files.
|
||||
Compressed and non-compress blobs of the same type may be mixed in a pack
|
||||
Compressed and non-compressed blobs of the same type may be mixed in a pack
|
||||
file.
|
||||
|
||||
For reconstructing the index or parsing a pack without an index, first
|
||||
@@ -237,7 +237,7 @@ it from plain JSON and to allow for further evolution of the storage format:
|
||||
The ``encoding_version`` field is encoded as one byte.
|
||||
For backwards compatibility the encoding versions '[' (0x5b) and '{' (0x7b)
|
||||
are used to mark that the whole plaintext (including the encoding version
|
||||
byte) should treated as JSON document.
|
||||
byte) should be treated as JSON document.
|
||||
|
||||
For new data the encoding version is currently always ``2``. For that
|
||||
version ``data`` contains a JSON document compressed using the zstandard
|
||||
@@ -314,8 +314,8 @@ counter mode and authenticated using Poly1305-AES. For encrypting new
|
||||
data first 16 bytes are read from a cryptographically secure
|
||||
pseudo-random number generator as a random nonce. This is used both as
|
||||
the IV for counter mode and the nonce for Poly1305. This operation needs
|
||||
three keys: A 32 byte for AES-256 for encryption, a 16 byte AES key and
|
||||
a 16 byte key for Poly1305. For details see the original paper `The
|
||||
three keys: a 32-byte key for AES-256 encryption, a 16-byte AES key and
|
||||
a 16-byte key for Poly1305. For details see the original paper `The
|
||||
Poly1305-AES message-authentication
|
||||
code <https://cr.yp.to/mac/poly1305-20050329.pdf>`__ by Dan Bernstein.
|
||||
The data is then encrypted with AES-256 and afterwards a message
|
||||
@@ -420,7 +420,7 @@ and pretty-print the contents of a snapshot file:
|
||||
|
||||
Here it can be seen that this snapshot represents the contents of the
|
||||
directory ``/tmp/testdata``. The most important field is ``tree``. When
|
||||
the meta data (e.g. the tags) of a snapshot change, the snapshot needs
|
||||
the metadata (e.g. the tags) of a snapshot change, the snapshot needs
|
||||
to be re-encrypted and saved. This will change the storage ID, so in
|
||||
order to relate these seemingly different snapshots, a field
|
||||
``original`` is introduced which contains the ID of the original
|
||||
@@ -449,7 +449,7 @@ becomes:
|
||||
}
|
||||
|
||||
Once introduced, the ``original`` field is not modified when the
|
||||
snapshot's meta data is changed again.
|
||||
snapshot's metadata is changed again.
|
||||
|
||||
All content within a restic repository is referenced according to its
|
||||
SHA-256 hash. Before saving, each file is split into variable sized
|
||||
@@ -499,7 +499,7 @@ the JSON is indented):
|
||||
}
|
||||
|
||||
A tree contains a list of entries (in the field ``nodes``) which contain
|
||||
meta data like a name and timestamps. Note that there are some specialties of how
|
||||
metadata like a name and timestamps. Note that there are some specialties of how
|
||||
this metadata is generated:
|
||||
|
||||
- The name is quoted using `strconv.Quote <https://pkg.go.dev/strconv#Quote>`__
|
||||
@@ -572,11 +572,11 @@ A symlink uses the following data structure:
|
||||
]
|
||||
}
|
||||
|
||||
The symlink target is stored in the field `linktarget`. As JSON strings can
|
||||
only contain valid unicode, an exception applies if the `linktarget` is not a
|
||||
valid UTF-8 string. Since restic 0.16.0, in such a case the `linktarget_raw`
|
||||
The symlink target is stored in the field ``linktarget``. As JSON strings can
|
||||
only contain valid unicode, an exception applies if the ``linktarget`` is not a
|
||||
valid UTF-8 string. Since restic 0.16.0, in such a case the ``linktarget_raw``
|
||||
field contains a base64 encoded version of the raw linktarget. The
|
||||
`linktarget_raw` field is only set if `linktarget` cannot be encoded correctly.
|
||||
``linktarget_raw`` field is only set if ``linktarget`` cannot be encoded correctly.
|
||||
|
||||
The command ``restic cat blob`` can also be used to extract and decrypt
|
||||
data given a plaintext ID, e.g. for the data mentioned above:
|
||||
@@ -595,7 +595,7 @@ Locks
|
||||
=====
|
||||
|
||||
The restic repository structure is designed in a way that allows
|
||||
parallel access of multiple instance of restic and even parallel writes.
|
||||
parallel access of multiple instances of restic and even parallel writes.
|
||||
However, there are some functions that work more efficient or even
|
||||
require exclusive access of the repository. In order to implement these
|
||||
functions, restic processes are required to create a lock on the
|
||||
|
||||
+8
-8
@@ -46,7 +46,7 @@ looks like this:
|
||||
pack 819a9a52e4f51230afa89aefbf90df37fb70996337ae57e6f7a822959206a85e: not referenced in any index
|
||||
pack de299e69fb075354a3775b6b045d152387201f1cdc229c31d1caa34c3b340141: not referenced in any index
|
||||
2 additional files were found in the repo, which likely contain duplicate data.
|
||||
You can run `restic prune` to correct this.
|
||||
You can run ``restic prune`` to correct this.
|
||||
check snapshots, trees and blobs
|
||||
[0:00] 100.00% 16 / 16 snapshots
|
||||
no errors were found
|
||||
@@ -101,7 +101,7 @@ Restic handles globbing and expansion in the following ways:
|
||||
- Environment variables are not expanded in the file read via ``--files-from``
|
||||
- ``*`` is expanded for paths read via ``--files-from``
|
||||
- e.g. For backup sources given to restic as arguments on the shell, neither glob expansion nor shell variable replacement is done. If restic is called as ``restic backup '*' '$HOME'``, it will try to backup the literal file(s)/dir(s) ``*`` and ``$HOME``
|
||||
- Double-asterisk ``**`` matches across zero or more subdirectories in exclude patterns (e.g. ``foo/**/bar``). This is a custom extension beyond Go's ``filepath.Match``; make sure the shell does not expand it (e.g. by quoting the pattern). See :ref:`backup-excluding-files` for details
|
||||
- Double-asterisk ``**`` matches across zero or more subdirectories in exclude/include patterns (not ``--files-from``) when it is a complete path component (e.g. ``foo/**/bar``, not ``foo**`` or ``foo/**bar``). This is a custom extension beyond Go's ``filepath.Match``; make sure the shell does not expand it (e.g. by quoting the pattern). See :ref:`backup-excluding-files` for details
|
||||
|
||||
|
||||
How can I specify encryption passwords automatically?
|
||||
@@ -280,17 +280,17 @@ Archive** storage classes is available:
|
||||
**Notes:**
|
||||
|
||||
- This feature is still in early alpha stage. Expect arbitrary breaking changes
|
||||
in the future (although we'll do our best-effort to avoid them).
|
||||
in the future (although the project will make best-effort attempts to avoid them).
|
||||
- Expect restores to hang from 1 up to 42 hours depending on your storage
|
||||
class, provider and luck. Restores from cold storages are known to be
|
||||
time-consuming. You may need to adjust the `s3.restore-timeout` if a restore
|
||||
time-consuming. You may need to adjust the ``s3.restore-timeout`` option if a restore
|
||||
operation takes more than 24 hours.
|
||||
- Restic will prevent sending metadata files (such as config files, lock files
|
||||
or tree blobs) to Glacier or Deep Archive. Standard class is used instead to
|
||||
ensure normal and fast operations for most tasks.
|
||||
- Currently, only the following commands are known to work:
|
||||
|
||||
- `backup`
|
||||
- `copy`
|
||||
- `prune`
|
||||
- `restore`
|
||||
- ``backup``
|
||||
- ``copy``
|
||||
- ``prune``
|
||||
- ``restore``
|
||||
|
||||
+18
-20
@@ -86,7 +86,7 @@ Usage help is available:
|
||||
|
||||
Similar to programs such as ``git``, restic has a number of
|
||||
sub-commands. You can see these commands in the listing above. Each
|
||||
sub-command may have own command-line options, and there is a help
|
||||
sub-command may have its own command-line options, and there is a help
|
||||
option for each command which lists them, e.g. for the ``backup``
|
||||
command:
|
||||
|
||||
@@ -180,19 +180,19 @@ Additionally, on Unix systems if ``restic`` receives a SIGUSR1 signal the
|
||||
current progress will be written to the standard output so you can check up
|
||||
on the status at will.
|
||||
|
||||
Setting the `RESTIC_PROGRESS_FPS` environment variable or sending a `SIGUSR1`
|
||||
signal prints a status report even when `--quiet` was specified.
|
||||
Setting the ``RESTIC_PROGRESS_FPS`` environment variable or sending a ``SIGUSR1``
|
||||
signal prints a status report even when ``--quiet`` was specified.
|
||||
|
||||
Manage tags
|
||||
-----------
|
||||
Managing tags
|
||||
-------------
|
||||
|
||||
Managing tags on snapshots is done with the ``tag`` command. The
|
||||
existing set of tags can be replaced completely, tags can be added or
|
||||
removed. The result is directly visible in the ``snapshots`` command.
|
||||
|
||||
Let's say we want to tag snapshot ``590c8fc8`` with the tags ``NL`` and
|
||||
``CH`` and remove all other tags that may be present, the following
|
||||
command does that:
|
||||
To tag snapshot ``590c8fc8`` with the tags ``NL`` and
|
||||
``CH`` and remove all other tags that may be present, run the following
|
||||
command:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -200,10 +200,10 @@ command does that:
|
||||
create exclusive lock for repository
|
||||
modified tags on 1 snapshots
|
||||
|
||||
Note the snapshot ID has changed, so between each change we need to look up the
|
||||
Note the snapshot ID has changed, so between each change you need to look up the
|
||||
new ID of the snapshot. But there is an even better way - the ``tag`` command
|
||||
accepts a filter using the ``--tag`` option, so we can filter snapshots based
|
||||
on the tag we just added. This way we can add and remove tags incrementally:
|
||||
accepts a filter using the ``--tag`` option, so you can filter snapshots based
|
||||
on the tag you just added. This way you can add and remove tags incrementally:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -253,9 +253,7 @@ repository.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ restic -r backup find test.txt
|
||||
debug log file restic.log
|
||||
debug enabled
|
||||
$ restic -r /srv/restic-repo find test.txt
|
||||
enter password for repository:
|
||||
found 1 matching entries in snapshot 196bc5760c909a7681647949e80e5448e276521489558525680acf1bd428af36
|
||||
-rw-r--r-- 501 20 5 2015-08-26 14:09:57 +0200 CEST path/to/test.txt
|
||||
@@ -356,15 +354,15 @@ host by using the ``--host`` flag:
|
||||
Total File Count: 21766
|
||||
Total Size: 481.783 GiB
|
||||
|
||||
There we see that it would take 482 GiB of disk space to restore the latest
|
||||
This shows that it would take 482 GiB of disk space to restore the latest
|
||||
snapshot from "myserver".
|
||||
|
||||
In case you have multiple backups running from the same host so can also use
|
||||
In case you have multiple backups running from the same host you can also use
|
||||
``--tag`` and ``--path`` to be more specific about which snapshots you
|
||||
are looking for.
|
||||
|
||||
But how much space does that snapshot take on disk? In other words, how much
|
||||
has restic's deduplication helped? We can check:
|
||||
has restic's deduplication helped? You can check:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
@@ -372,7 +370,7 @@ has restic's deduplication helped? We can check:
|
||||
Total Blob Count: 340847
|
||||
Total Size: 458.663 GiB
|
||||
|
||||
Comparing this size to the previous command, we see that restic has saved
|
||||
Comparing this size to the previous command, you can see that restic has saved
|
||||
about 23 GiB of space with deduplication.
|
||||
|
||||
Which mode you use depends on your exact use case. Some modes are more useful
|
||||
@@ -444,7 +442,7 @@ Caching
|
||||
-------
|
||||
|
||||
Restic keeps a cache with some files from the repository on the local machine.
|
||||
This allows faster operations, since meta data does not need to be loaded from
|
||||
This allows faster operations, since metadata does not need to be loaded from
|
||||
a remote repository. The cache is automatically created, usually in an
|
||||
OS-specific cache folder:
|
||||
|
||||
@@ -457,7 +455,7 @@ If the relevant environment variables are not set, restic exits with an error
|
||||
message.
|
||||
|
||||
The command line parameter ``--cache-dir`` or the environment variable
|
||||
``$RESTIC_CACHE_DIR`` can be used to override the default cache location. The
|
||||
``$RESTIC_CACHE_DIR`` can be used to override the default cache location. The
|
||||
parameter ``--no-cache`` disables the cache entirely. In this case, all data
|
||||
is loaded from the repository.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user