mirror of
https://github.com/restic/restic.git
synced 2026-11-07 07:49:21 +00:00
Convert AWS S3 tutorial to rst
This commit is contained in:
Alexander Neumann committed
1 parent
adb602dbfe
commit
aafee25d58
20 files changed
+261
-216
No files matched your search
@@ -1,216 +0,0 @@
|
||||
# Setting up Restic with Amazon S3
|
||||
|
||||
## Preface
|
||||
|
||||
This tutorial will show you how to use Restic with AWS S3. It will show you how
|
||||
to navigate the AWS web interface, create an S3 bucket, create a user with
|
||||
access to only this bucket, and finally how to connect Restic to this bucket.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
You should already have a `restic` binary available on your system that you can
|
||||
run. Furthermore, you should also have an account with
|
||||
[AWS](https://aws.amazon.com/). You will likely need to provide credit card
|
||||
details for billing purposes, even if you use their
|
||||
[free-tier](https://aws.amazon.com/free/).
|
||||
|
||||
|
||||
## Logging into AWS
|
||||
|
||||
Point your browser to
|
||||
https://console.aws.amazon.com
|
||||
and log in using your AWS account. You will be presented with the AWS homepage:
|
||||
|
||||
|
||||
By using the "Services" button in the upper left corder, a menu of all services
|
||||
provided by AWS can be opened:
|
||||
|
||||
|
||||
For this tutorial, the Simple Storage Service (S3), as well as Identity and
|
||||
Access Management (IAM) are relevant.
|
||||
|
||||
|
||||
## Creating the bucket
|
||||
|
||||
First, a bucket to store your backups in must be created. Using the "Services"
|
||||
menu, navigate to S3. In case you already have some S3 buckets, you will see a
|
||||
list of them here:
|
||||
|
||||
|
||||
Click the "Create bucket" button and choose a name and region for your new
|
||||
bucket. For the purpose of this tutorial, the bucket will be named
|
||||
`restic-demo` and reside in Frankfurk. Because the bucket namespace is shared among all AWS users, the
|
||||
name `restic-demo` may not be available to you. Be creative and choose a unique
|
||||
bucket name.
|
||||
|
||||
|
||||
It is not necessary to configure any special properties or permissions of the
|
||||
bucket just yet. Therefore, just finish the wizard without making any further
|
||||
changes:
|
||||
|
||||
|
||||
The newly created `restic-demo` bucket will no appear on the list of S3
|
||||
buckets:
|
||||
|
||||
|
||||
|
||||
## Creating a user
|
||||
|
||||
Use the "Services" menu of the AWS web interface to navigate to IAM. This will
|
||||
bring you to the IAM homepage. To create a new user, click on the "Users" menu
|
||||
entry on the left:
|
||||
|
||||
|
||||
In case you already have set-up users with IAM before, you will see a list of
|
||||
them here. Use the "Add user" button at the top to create a new user:
|
||||
|
||||
|
||||
For this tutorial, the new user will be named `restic-demo-user`. Feel free to
|
||||
choose your own name that best fits your needs. This user will only ever access
|
||||
AWS through the `restic` program and not through the web interface. Therefore,
|
||||
"Programmatic access" is selected for "Access type":
|
||||
|
||||
|
||||
During the next step, permissions can be assigned to the new user. To use this
|
||||
user with Restic, it only needs access to the `restic-demo` bucket. Select
|
||||
"Attach exiting policies directly", which will bring up a list of pre-defined
|
||||
policies below. Afterwards, click the "Create policy" button to create a custom
|
||||
policy:
|
||||
|
||||
|
||||
A new browser window or tab will open with the policy wizard. In Amazon IAM,
|
||||
policies are defined as JSON documents. For this tutorial, the "Policy
|
||||
Generator" will be used to generate a policy file using a web interface:
|
||||
|
||||
|
||||
After invoking the policy generator, you will be presented with a user
|
||||
interface to generate individual permission statements. For Restic to work, two
|
||||
such statements must be created. The first statement is set up as follows:
|
||||
|
||||
```
|
||||
Effect: Allow
|
||||
Service: S3
|
||||
Actions: DeleteObject, GetObject, PutObject
|
||||
Resource: arn:aws:s3:::restic-demo/*
|
||||
```
|
||||
|
||||
This statement allows Restic to create, read and delete objects inside the S3
|
||||
bucket named `restic-demo`. Adjust the bucket's name to the name of the bucket
|
||||
you created earlier. Using the "Add Statement" button, this statement can be
|
||||
saved. Now a second statement is created:
|
||||
|
||||
```
|
||||
Effect: Allow
|
||||
Service: S3
|
||||
Actions: ListBucket
|
||||
Resource: arn:aws:s3:::restic-demo
|
||||
```
|
||||
|
||||
Again, substitute `restic-demo` with the actual name of your bucket. Note that,
|
||||
unlike before, there is no `/*` after the bucket name. This statement allows
|
||||
Restic to list the objects stored in the `restic-demo` bucket. Again, use "Add
|
||||
Statement" to save this statement. The policy creator interface should now
|
||||
look as follows:
|
||||
|
||||
|
||||
Continue to the next step and enter a name and description for this policy. For
|
||||
this tutorial, the policy will be named `restic-demo-policy`. In this step you
|
||||
can also examine the JSON document created by the policy generator. Click
|
||||
"Create Policy" to finish the process:
|
||||
|
||||
|
||||
Go back to the browser window or tab where you were previously creating the new
|
||||
user. Click the button labeled "Refresh" above the list of policies to make
|
||||
sure the newly created policy is available to you. Afterwards, use the search
|
||||
function to search for the `restic-demo-policy`. Select this policy using the
|
||||
checkbox on the left. Then, continue to the next step.
|
||||
|
||||
|
||||
The next page will present an overview of the user account that is about to be
|
||||
created. If everything looks good, click "Create user" to complete the process:
|
||||
|
||||
|
||||
After the user has been created, its access credentials will be displayed. They
|
||||
consist of the "Access key ID" (think username), and the "Secret access key"
|
||||
(think password). Copy these down to a safe place.
|
||||
|
||||
|
||||
You have now completed the configuration in AWS. Feel free to close your web
|
||||
browser now.
|
||||
|
||||
|
||||
## Initializing the Restic repository
|
||||
|
||||
Open a terminal and make sure you have the `restic` binary ready. First, choose
|
||||
a password to encrypt your backups with. In this tutorial, `apg` is used for
|
||||
this purpose:
|
||||
|
||||
```console
|
||||
$ apg -a 1 -m 32 -n 1 -M NCL
|
||||
I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5
|
||||
```
|
||||
|
||||
Note this password somewhere safe along with your AWS credentials. Next, the
|
||||
configuration of Restic will be placed into environment variables. This will
|
||||
include sensitive information, such as your AWS secret and repository password.
|
||||
Therefore, make sure the next commands **do not** end up in your shell's
|
||||
history file. Adjust the contents of the environment variables to fit your
|
||||
bucket's name and your user's API credentials.
|
||||
|
||||
```console
|
||||
$ unset HISTFILE
|
||||
$ export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/restic-demo"
|
||||
$ export AWS_ACCESS_KEY_ID="AKIAJAJSLTZCAZ4SRI5Q"
|
||||
$ export AWS_SECRET_ACCESS_KEY="LaJtZPoVvGbXsaD2LsxvJZF/7LRi4FhT0TK4gDQq"
|
||||
$ export RESTIC_PASSWORD="I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5"
|
||||
```
|
||||
|
||||
After the environment is set up, Restic may be called to initialize the
|
||||
repository:
|
||||
|
||||
|
||||
```console
|
||||
$ ./restic init
|
||||
created restic backend b5c661a86a at s3:https://s3.amazonaws.com/restic-demo
|
||||
|
||||
Please note that knowledge of your password is required to access
|
||||
the repository. Losing your password means that your data is
|
||||
irrecoverably lost.
|
||||
```
|
||||
|
||||
Restic is now ready to be used with AWS S3. Try to create a backup:
|
||||
|
||||
```console
|
||||
$ dd if=/dev/urandom bs=1M count=10 of=test.bin
|
||||
10+0 records in
|
||||
10+0 records out
|
||||
10485760 bytes (10 MB, 10 MiB) copied, 0,0891322 s, 118 MB/s
|
||||
|
||||
$ ./restic backup test.bin
|
||||
scan [/home/philip/restic-demo/test.bin]
|
||||
scanned 0 directories, 1 files in 0:00
|
||||
[0:04] 100.00% 2.500 MiB/s 10.000 MiB / 10.000 MiB 1 / 1 items ... ETA 0:00
|
||||
duration: 0:04, 2.47MiB/s
|
||||
snapshot 10fdbace saved
|
||||
|
||||
$ ./restic snapshots
|
||||
ID Date Host Tags Directory
|
||||
----------------------------------------------------------------------
|
||||
10fdbace 2017-03-26 16:41:50 blackbox /home/philip/restic-demo/test.bin
|
||||
```
|
||||
|
||||
A snapshot was created and stored in the S3 bucket. This snapshot may now be
|
||||
restored:
|
||||
|
||||
```console
|
||||
$ mkdir restore
|
||||
|
||||
$ ./restic restore 10fdbace --target restore
|
||||
restoring <Snapshot 10fdbace of [/home/philip/restic-demo/test.bin] at 2017-03-26 16:41:50.201418102 +0200 CEST by philip@blackbox> to restore
|
||||
|
||||
$ ls restore/
|
||||
test.bin
|
||||
```
|
||||
|
||||
The snapshot was successfully restored. This concludes the tutorial.
|
||||
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -7,6 +7,7 @@ Restic Documentation
|
||||
installation
|
||||
manual
|
||||
faq
|
||||
tutorials
|
||||
development
|
||||
references
|
||||
talks
|
||||
|
||||
@@ -0,0 +1,255 @@
|
||||
Setting up Restic with Amazon S3
|
||||
================================
|
||||
|
||||
Preface
|
||||
-------
|
||||
|
||||
This tutorial will show you how to use Restic with AWS S3. It will show you how
|
||||
to navigate the AWS web interface, create an S3 bucket, create a user with
|
||||
access to only this bucket, and finally how to connect Restic to this bucket.
|
||||
|
||||
Prerequisites
|
||||
-------------
|
||||
|
||||
You should already have a ``restic`` binary available on your system that you can
|
||||
run. Furthermore, you should also have an account with
|
||||
`AWS <https://aws.amazon.com/>`__. You will likely need to provide credit card
|
||||
details for billing purposes, even if you use their
|
||||
`free-tier <https://aws.amazon.com/free/>`__.
|
||||
|
||||
|
||||
Logging into AWS
|
||||
----------------
|
||||
|
||||
Point your browser to
|
||||
https://console.aws.amazon.com
|
||||
and log in using your AWS account. You will be presented with the AWS homepage:
|
||||
|
||||
.. image:: images/aws_s3/01_aws_start.png
|
||||
:alt: AWS Homepage
|
||||
|
||||
By using the "Services" button in the upper left corder, a menu of all services
|
||||
provided by AWS can be opened:
|
||||
|
||||
.. image:: images/aws_s3/02_aws_menu.png
|
||||
:alt: AWS Services Menu
|
||||
|
||||
For this tutorial, the Simple Storage Service (S3), as well as Identity and
|
||||
Access Management (IAM) are relevant.
|
||||
|
||||
|
||||
Creating the bucket
|
||||
-------------------
|
||||
|
||||
First, a bucket to store your backups in must be created. Using the "Services"
|
||||
menu, navigate to S3. In case you already have some S3 buckets, you will see a
|
||||
list of them here:
|
||||
|
||||
.. image:: images/aws_s3/03_buckets_list_before.png
|
||||
:alt: List of S3 Buckets
|
||||
|
||||
Click the "Create bucket" button and choose a name and region for your new
|
||||
bucket. For the purpose of this tutorial, the bucket will be named
|
||||
``restic-demo`` and reside in Frankfurk. Because the bucket name space is
|
||||
shared among all AWS users, the name ``restic-demo`` may not be available to
|
||||
you. Be creative and choose a unique bucket name.
|
||||
|
||||
.. image:: images/aws_s3/04_bucket_create_start.png
|
||||
:alt: Create a Bucket
|
||||
|
||||
It is not necessary to configure any special properties or permissions of the
|
||||
bucket just yet. Therefore, just finish the wizard without making any further
|
||||
changes:
|
||||
|
||||
.. image:: images/aws_s3/05_bucket_create_review.png
|
||||
:alt: Review Bucket Creation
|
||||
|
||||
The newly created ``restic-demo`` bucket will no appear on the list of S3
|
||||
buckets:
|
||||
|
||||
.. image:: images/aws_s3/06_buckets_list_after.png
|
||||
:alt: List With New Bucket
|
||||
|
||||
Creating a user
|
||||
---------------
|
||||
|
||||
Use the "Services" menu of the AWS web interface to navigate to IAM. This will
|
||||
bring you to the IAM homepage. To create a new user, click on the "Users" menu
|
||||
entry on the left:
|
||||
|
||||
.. image:: images/aws_s3/07_iam_start.png
|
||||
:alt: IAM Home Page
|
||||
|
||||
In case you already have set-up users with IAM before, you will see a list of
|
||||
them here. Use the "Add user" button at the top to create a new user:
|
||||
|
||||
.. image:: images/aws_s3/08_user_list.png
|
||||
:alt: IAM User List
|
||||
|
||||
For this tutorial, the new user will be named ``restic-demo-user``. Feel free to
|
||||
choose your own name that best fits your needs. This user will only ever access
|
||||
AWS through the ``restic`` program and not through the web interface. Therefore,
|
||||
"Programmatic access" is selected for "Access type":
|
||||
|
||||
.. image:: images/aws_s3/09_user_name.png
|
||||
:alt: Choose User Name and Access Type
|
||||
|
||||
During the next step, permissions can be assigned to the new user. To use this
|
||||
user with Restic, it only needs access to the ``restic-demo`` bucket. Select
|
||||
"Attach exiting policies directly", which will bring up a list of pre-defined
|
||||
policies below. Afterwards, click the "Create policy" button to create a custom
|
||||
policy:
|
||||
|
||||
.. image:: images/aws_s3/10_user_pre_policy.png
|
||||
:alt: Assign a Policy
|
||||
|
||||
A new browser window or tab will open with the policy wizard. In Amazon IAM,
|
||||
policies are defined as JSON documents. For this tutorial, the "Policy
|
||||
Generator" will be used to generate a policy file using a web interface:
|
||||
|
||||
.. image:: images/aws_s3/11_policy_start.png
|
||||
:alt: Create a New Policy
|
||||
|
||||
After invoking the policy generator, you will be presented with a user
|
||||
interface to generate individual permission statements. For Restic to work, two
|
||||
such statements must be created. The first statement is set up as follows:
|
||||
|
||||
.. code::
|
||||
|
||||
Effect: Allow
|
||||
Service: S3
|
||||
Actions: DeleteObject, GetObject, PutObject
|
||||
Resource: arn:aws:s3:::restic-demo/*
|
||||
|
||||
This statement allows Restic to create, read and delete objects inside the S3
|
||||
bucket named ``restic-demo``. Adjust the bucket's name to the name of the bucket
|
||||
you created earlier. Using the "Add Statement" button, this statement can be
|
||||
saved. Now a second statement is created:
|
||||
|
||||
.. code::
|
||||
|
||||
Effect: Allow
|
||||
Service: S3
|
||||
Actions: ListBucket
|
||||
Resource: arn:aws:s3:::restic-demo
|
||||
|
||||
Again, substitute ``restic-demo`` with the actual name of your bucket. Note that,
|
||||
unlike before, there is no ``/*`` after the bucket name. This statement allows
|
||||
Restic to list the objects stored in the ``restic-demo`` bucket. Again, use "Add
|
||||
Statement" to save this statement. The policy creator interface should now
|
||||
look as follows:
|
||||
|
||||
.. image:: images/aws_s3/12_policy_permissions_done.png
|
||||
:alt: Policy Creator With Two Statements
|
||||
|
||||
Continue to the next step and enter a name and description for this policy. For
|
||||
this tutorial, the policy will be named ``restic-demo-policy``. In this step you
|
||||
can also examine the JSON document created by the policy generator. Click
|
||||
"Create Policy" to finish the process:
|
||||
|
||||
.. image:: images/aws_s3/13_policy_review.png
|
||||
:alt: Policy Review
|
||||
|
||||
Go back to the browser window or tab where you were previously creating the new
|
||||
user. Click the button labeled "Refresh" above the list of policies to make
|
||||
sure the newly created policy is available to you. Afterwards, use the search
|
||||
function to search for the ``restic-demo-policy``. Select this policy using the
|
||||
checkbox on the left. Then, continue to the next step.
|
||||
|
||||
.. image:: images/aws_s3/14_user_attach_policy.png
|
||||
:alt: Attach Policy to User
|
||||
|
||||
The next page will present an overview of the user account that is about to be
|
||||
created. If everything looks good, click "Create user" to complete the process:
|
||||
|
||||
.. image:: images/aws_s3/15_user_review.png
|
||||
:alt: User Creation Review
|
||||
|
||||
After the user has been created, its access credentials will be displayed. They
|
||||
consist of the "Access key ID" (think user name), and the "Secret access key"
|
||||
(think password). Copy these down to a safe place.
|
||||
|
||||
.. image:: images/aws_s3/16_user_created.png
|
||||
:alt: User Credentials
|
||||
|
||||
You have now completed the configuration in AWS. Feel free to close your web
|
||||
browser now.
|
||||
|
||||
|
||||
Initializing the Restic repository
|
||||
----------------------------------
|
||||
|
||||
Open a terminal and make sure you have the ``restic`` binary ready. First, choose
|
||||
a password to encrypt your backups with. In this tutorial, ``apg`` is used for
|
||||
this purpose:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ apg -a 1 -m 32 -n 1 -M NCL
|
||||
I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5
|
||||
|
||||
Note this password somewhere safe along with your AWS credentials. Next, the
|
||||
configuration of Restic will be placed into environment variables. This will
|
||||
include sensitive information, such as your AWS secret and repository password.
|
||||
Therefore, make sure the next commands **do not** end up in your shell's
|
||||
history file. Adjust the contents of the environment variables to fit your
|
||||
bucket's name and your user's API credentials.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ unset HISTFILE
|
||||
$ export RESTIC_REPOSITORY="s3:https://s3.amazonaws.com/restic-demo"
|
||||
$ export AWS_ACCESS_KEY_ID="AKIAJAJSLTZCAZ4SRI5Q"
|
||||
$ export AWS_SECRET_ACCESS_KEY="LaJtZPoVvGbXsaD2LsxvJZF/7LRi4FhT0TK4gDQq"
|
||||
$ export RESTIC_PASSWORD="I9n7G7G0ZpDWA3GOcJbIuwQCGvGUBkU5"
|
||||
|
||||
|
||||
After the environment is set up, Restic may be called to initialize the
|
||||
repository:
|
||||
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ ./restic init
|
||||
created restic backend b5c661a86a at s3:https://s3.amazonaws.com/restic-demo
|
||||
|
||||
Please note that knowledge of your password is required to access
|
||||
the repository. Losing your password means that your data is
|
||||
irrecoverably lost.
|
||||
|
||||
Restic is now ready to be used with AWS S3. Try to create a backup:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ dd if=/dev/urandom bs=1M count=10 of=test.bin
|
||||
10+0 records in
|
||||
10+0 records out
|
||||
10485760 bytes (10 MB, 10 MiB) copied, 0,0891322 s, 118 MB/s
|
||||
|
||||
$ ./restic backup test.bin
|
||||
scan [/home/philip/restic-demo/test.bin]
|
||||
scanned 0 directories, 1 files in 0:00
|
||||
[0:04] 100.00% 2.500 MiB/s 10.000 MiB / 10.000 MiB 1 / 1 items ... ETA 0:00
|
||||
duration: 0:04, 2.47MiB/s
|
||||
snapshot 10fdbace saved
|
||||
|
||||
$ ./restic snapshots
|
||||
ID Date Host Tags Directory
|
||||
----------------------------------------------------------------------
|
||||
10fdbace 2017-03-26 16:41:50 blackbox /home/philip/restic-demo/test.bin
|
||||
|
||||
A snapshot was created and stored in the S3 bucket. This snapshot may now be
|
||||
restored:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ mkdir restore
|
||||
|
||||
$ ./restic restore 10fdbace --target restore
|
||||
restoring <Snapshot 10fdbace of [/home/philip/restic-demo/test.bin] at 2017-03-26 16:41:50.201418102 +0200 CEST by philip@blackbox> to restore
|
||||
|
||||
$ ls restore/
|
||||
test.bin
|
||||
|
||||
The snapshot was successfully restored. This concludes the tutorial.
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
==========
|
||||
Tutorials
|
||||
==========
|
||||
|
||||
.. include:: tutorial_aws_s3.rst
|
||||
Reference in new issue
Block a user