Merge pull request #3731 from metalsp0rk/feature/min-packsize-flag

Feature: min packsize flag
This commit is contained in:
MichaelEischer authored and GitHub committed 2022-08-07 14:54:45 +02:00
commit 2930a102de
17 files changed
+191 -34

No files matched your search

+1
View File
@@ -554,6 +554,7 @@ environment variables. The following lists these environment variables:
RESTIC_CACHE_DIR Location of the cache directory
RESTIC_COMPRESSION Compression mode (only available for repository format version 2)
RESTIC_PROGRESS_FPS Frames per second by which the progress bar is updated
RESTIC_PACK_SIZE Target size for pack files
TMPDIR Location for temporary files
+24
View File
@@ -8,6 +8,7 @@
- for subsections
^ for subsubsections
" for paragraphs
########################
Tuning Backup Parameters
########################
@@ -48,3 +49,26 @@ which will compress very fast), ``max`` (which will trade backup speed and CPU u
slightly better compression), or ``off`` (which disables compression). Each setting is
only applied for the single run of restic. The option can also be set via the environment
variable ``RESTIC_COMPRESSION``.
Pack Size
=========
In certain instances, such as very large repositories (in the TiB range) or very fast
upload connections, it is desirable to use larger pack sizes to reduce the number of
files in the repository and improve upload performance. Notable examples are OpenStack
Swift and some Google Drive Team accounts, where there are hard limits on the total
number of files. Larger pack sizes can also improve the backup speed for a repository
stored on a local HDD. This can be achieved by either using the ``--pack-size`` option
or defining the ``$RESTIC_PACK_SIZE`` environment variable. Restic currently defaults
to a 16 MiB pack size.
The side effect of increasing the pack size is requiring more disk space for temporary pack
files created before uploading. The space must be available in the system default temp
directory, unless overwritten by setting the ``$TMPDIR`` environment variable. In addition,
depending on the backend the memory usage can also increase by a similar amount. Restic
requires temporary space according to the pack size, multiplied by the number
of backend connections plus one. For example, if the backend uses 5 connections (the default
for most backends), with a target pack size of 64 MiB, you'll need a *minimum* of 384 MiB
of space in the temp directory. A bit of tuning may be required to strike a balance between
resource usage at the backup client and the number of pack files in the repository.
+2
View File
@@ -56,6 +56,7 @@ Usage help is available:
--key-hint key key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
--limit-download int limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload int limits uploads to a maximum rate in KiB/s. (default: unlimited)
--pack-size uint set target pack size in MiB. (default: $RESTIC_PACK_SIZE)
--no-cache do not use a local cache
--no-lock do not lock the repository, this allows some operations on read-only repositories
-o, --option key=value set extended option (key=value, can be specified multiple times)
@@ -128,6 +129,7 @@ command:
--key-hint key key ID of key to try decrypting first (default: $RESTIC_KEY_HINT)
--limit-download int limits downloads to a maximum rate in KiB/s. (default: unlimited)
--limit-upload int limits uploads to a maximum rate in KiB/s. (default: unlimited)
--pack-size uint set target pack size in MiB. (default: $RESTIC_PACK_SIZE)
--no-cache do not use a local cache
--no-lock do not lock the repository, this allows some operations on read-only repositories
-o, --option key=value set extended option (key=value, can be specified multiple times)