backup: Add --ignore-ctime option and document change detection

doc/040_backup.rst

@@ -131,24 +131,62 @@ restic encounters:
 In fact several hosts may use the same repository to backup directories
 and files leading to a greater de-duplication.
 
-Please be aware that when you backup different directories (or the
-directories to be saved have a variable name component like a
-time/date), restic always needs to read all files and only afterwards
-can compute which parts of the files need to be saved. When you backup
-the same directory again (maybe with new or changed files) restic will
-find the old snapshot in the repo and by default only reads those files
-that are new or have been modified since the last snapshot. This is
-decided based on the following attributes of the file in the file system:
-
- * Type (file, symlink, or directory?)
- * Modification time
- * Size
- * Inode number (internal number used to reference a file in a file system)
-
 Now is a good time to run ``restic check`` to verify that all data
 is properly stored in the repository. You should run this command regularly
 to make sure the internal structure of the repository is free of errors.
 
+File change detection
+*********************
+
+When restic encounters a file that has already been backed up, whether in the
+current backup or a previous one, it makes sure the file's contents are only
+stored once in the repository. To do so, it normally has to scan the entire
+contents of every file. Because this can be very expensive, restic also uses a
+change detection rule based on file metadata to determine whether a file is
+likely unchanged since a previous backup. If it is, the file is not scanned
+again.
+
+Change detection is only performed for regular files (not special files,
+symlinks or directories) that have the exact same path as they did in a
+previous backup of the same location.  If a file or one of its containing
+directories was renamed, it is considered a different file and its entire
+contents will be scanned again.
+
+Metadata changes (permissions, ownership, etc.) are always included in the
+backup, even if file contents are considered unchanged.
+
+On **Unix** (including Linux and Mac), given that a file lives at the same
+location as a file in a previous backup, the following file metadata
+attributes have to match for its contents to be presumed unchanged:
+
+ * Modification timestamp (mtime).
+ * Metadata change timestamp (ctime).
+ * File size.
+ * Inode number (internal number used to reference a file in a filesystem).
+
+The reason for requiring both mtime and ctime to match is that Unix programs
+can freely change mtime (and some do). In such cases, a ctime change may be
+the only hint that a file did change.
+
+The following ``restic backup`` command line flags modify the change detection
+rules:
+
+ * ``--force``: turn off change detection and rescan all files.
+ * ``--ignore-ctime``: require mtime to match, but allow ctime to differ.
+ * ``--ignore-inode``: require mtime to match, but allow inode number
+   and ctime to differ.
+
+The option ``--ignore-inode`` exists to support FUSE-based filesystems and
+pCloud, which do not assign stable inodes to files.
+
+Note that the device id of the containing mount point is never taken into
+account. Device numbers are not stable for removable devices and ZFS snapshots.
+If you want to force a re-scan in such a case, you can change the mountpoint.
+
+On **Windows**, a file is considered unchanged when its path and modification
+time match, and only ``--force`` has any effect. The other options are
+recognized but ignored.
+
 Excluding Files
 ***************
 
@@ -372,10 +410,6 @@ written, and the next backup needs to write new metadata again. If you really
 want to save the access time for files and directories, you can pass the
 ``--with-atime`` option to the ``backup`` command.
 
-In filesystems that do not support inode consistency, like FUSE-based ones and pCloud, it is
-possible to ignore inode on changed files comparison by passing ``--ignore-inode`` to
-``backup`` command.
-
 Reading data from stdin
 ***********************