Transferring your data from Dedibox to Elastic Metal

• compute
• dedibox
• elastic-metal
• transfer
• migration

Migrating your data from a Dedibox to an Elastic Metal server

This tutorial provides information about how to migrate your existing data from a Dedibox to an Elastic Metal server. Its purpose is to guide you in your migration to the resources that best fit your needs for improved stability, performance, and reliability.

We use Duplicity to encrypt the backup and to upload it to Object Storage. Then we download and decrypt the data on the Elastic Metal server.

Before you start

To complete the actions presented below, you must have:

• A Scaleway account logged into the console
• Owner status or IAM permissions allowing you to perform actions in the intended Organization
• An SSH key
• A Dedibox server
• Created and installed an Elastic Metal server

Creating an Object Storage bucket

1. Log in to the Scaleway console.
2. Click Storage on the side menu. A list of your buckets appears. If you have not created a bucket yet, the list will be empty.
3. Click Create a Bucket.
4. Name your bucket and validate your bucket creation. The bucket name must be unique and contain only alphanumeric and lowercase characters.

Installing software requirements on the Dedibox server

Run the following command to update the APT package manager, upgrade the software installed already installed on the server and to download and install Duplicity:

apt update && apt upgrade
apt install -y python3-boto python3-pip haveged gettext librsync-dev
wget https://gitlab.com/duplicity/duplicity/-/archive/rel.2.2.2/duplicity-rel.2.2.2.tar.gz
tar xaf duplicity-2.2.*.tar.gz
cd duplicity-2.2.*/
python3 -m pip install -r requirements.txt
python3 -m pip install

Creating a GPG key

1. To generate the GPG key, launch this command.

   gpg --full-generate-key
   

   Enter a passphrase. You will be asked to define the characteristics of your key. We will go with default settings:

   • What kind of key you want: (1) RSA and RSA (default)
   • What keysize do you want: (3072)
   • How long the key should be valid: 0 = key does not expire
   • GPG will then ask how to call your key, an address, and a description.

2. You need to use the GPG Key fingerprint, it could be an 8, 16 or 40 char long hash. You can also find the fingerprint of your key with the command:

   gpg --list-keys
   gpg: checking the trustdb
   gpg: marginals needed: 3  completes needed: 1  trust model: pgp
   gpg: depth: 0  valid: 1  signed: 0  trust: 0-, 0q, 0n, 0m, 0f, 1u
   /home/me/.gnupg/pubring.kbx
   ------------------------------
   pub   rsa3072 2022-01-25 [SC]
   XXXXXXXXXXXXX-FINGERPRINT-XXXXXXXXXXXXXX
   uid         [ultimate] backups (Scaleway Object Storage backups) <me@scaleway.com>
   sub   rsa3072 2022-01-25 [E]
   

Transferring the PGP key to the Elastic Metal server

1. Export the keys so you can decrypt your files on the Elastic Metal server. Also having the GPG private and public keys stored somewhere else will come in handy in case you lose access to your machine. Export the GPG keys with:

   gpg --export-secret-key keyname > ~/my-key.asc
   

2. Transfer the key to the Elastic Metal server using rsync.

   scp /root/my-key.asc root@<elastic-metal>:/root/
   

Backing up your Dedibox

1. Create the required folders and a configuration file for Duplicity:

   touch scw-backup.sh .scw-configrc
   chmod 700 scw-backup.sh
   chmod 600 .scw-configrc
   mkdir -p /var/log/duplicity
   touch /var/log/duplicity/logfile{.log,-recent.log}
   

2. Add the following lines to .scw-configrc:

   # Scaleway credentials keys
   export AWS_ACCESS_KEY_ID="<SCALEWAY ACCESS KEY>"
   export AWS_SECRET_ACCESS_KEY="<SCALEWAY SECRET ACCESS KEY>"
   export SCW_BUCKET="s3://s3.fr-par.scw.cloud/<NAME OF YOUR BUCKET>"
   # GPG Key information
   export PASSPHRASE="<YOUR GPG KEY PASSPHRASE>"
   export GPG_FINGERPRINT="<YOUR GPG KEY FINGERPRINT>"
   # Folder to back up
   export SOURCE="<PATH TO FOLDER TO BACKUP>"
   # Log files
   export LOGFILE_RECENT="/var/log/duplicity/logfile-recent.log"
   export LOGFILE="/var/log/duplicity/logfile.log"
   log () {
     date=`date +%Y-%m-%d`
     hour=`date +%H:%M:%S`
     echo "$date $hour $*" >> ${LOGFILE_RECENT}
   }
   export -f log
   

3. Copy the following script to scw-backup.sh:

   #!/bin/bash
   source <FULL PATH TO>/.scw-configrc
   currently_backuping=$(ps -ef | grep duplicity  | grep python | wc -l)
   if [ $currently_backuping -eq 0 ]; then
     # Clear the recent log file
     cat /dev/null > ${LOGFILE_RECENT}
     log ">>> creating and uploading backup to Object Storage"
     duplicity \
     full \
     --asynchronous-upload \
     --encrypt-key=${GPG_FINGERPRINT} \
     --sign-key=${GPG_FINGERPRINT} \
     ${SOURCE} ${SCW_BUCKET} >> ${LOGFILE_RECENT} 2>&1
     cat ${LOGFILE_RECENT} >> ${LOGFILE}
   fi
   

4. Run the backup:

   ./scw-backup.sh
   

5. Check if everything went well:

   cat /var/log/duplicity/logfile-recent.log
   

Restoring data on your Elastic Metal server

1. Install the required prerequisites and duplicity on the Elastic Metal server:

   apt update && apt upgrade
   apt install -y python3-boto python3-pip gettext librsync-dev
   wget https://launchpad.net/duplicity/0.8-series/0.8.21/+download/duplicity-0.8.21.tar.gz
   tar xaf duplicity-0.8.*.tar.gz
   cd duplicity-0.8.*/
   pip3 install -r requirements.txt
   python3 setup.py install
   

2. Import the GPG key on the Elastic Metal server:

   gpg --import my-key.asc
   

3. Create the required folders and a configuration file for Duplicity:

   touch scw-restore.sh .scw-configrc
   chmod 700 scw-restore.sh
   chmod 600 .scw-configrc
   mkdir -p /var/log/duplicity
   touch /var/log/duplicity/logfile{.log,-recent.log}
   

4. Add the following lines to .scw-configrc:

   # Scaleway credentials keys
   export AWS_ACCESS_KEY_ID="<SCALEWAY ACCESS KEY>"
   export AWS_SECRET_ACCESS_KEY="<SCALEWAY SECRET ACCESS KEY>"
   export SCW_BUCKET="s3://s3.fr-par.scw.cloud/<NAME OF YOUR BUCKET>"
   # GPG Key information
   export PASSPHRASE="<YOUR GPG KEY PASSPHRASE>"
   export GPG_FINGERPRINT="<YOUR GPG KEY FINGERPRINT>"
   # Folder to back up
   export SOURCE="<PATH TO FOLDER TO BACKUP>"
   # Log files
   export LOGFILE_RECENT="/var/log/duplicity/logfile-recent.log"
   export LOGFILE="/var/log/duplicity/logfile.log"
   log () {
     date=`date +%Y-%m-%d`
     hour=`date +%H:%M:%S`
     echo "$date $hour $*" >> ${LOGFILE_RECENT}
   }
   export -f log
   

5. Edit the file scw-restore.sh, and add the following:

   #!/bin/bash
   source .scw-configrc
   echo -e "Downloading full backup to:" $1
   duplicity \
     --time 0D \
     ${SCW_BUCKET} $1
   

6. Download the data from your bucket to the Elastic Metal server:

   ./scw-restore.sh /tmp/backup-recovery/
   

Once downloaded you can move the data to its final destination onto your new machine.