How to migrate your data from C2 and ARM64 to Virtual Instances

Overview

Scaleway C2 and ARM64 instances will reach end of life on December 1st, 2020 and support for these instances will end on July 1st, 2020. The purpose of this tutorial is to guide you in your migration to the resources that best fit your needs for improved stability, performance and reliability.

The Scaleway Elements public cloud platform provides two Virtual Instances ranges and Bare Metal cloud servers adapted to your use cases:

Requirements

Two ways of migration are recommended:

Solution 1 - Migrating Using Snapshots

Snapshots facilitate the migration of C2 instances with smaller amounts of data. It can be done within a few simple clicks in the Scaleway Console. If you are using ARM64 instances or want to benefit from upgrading the Operating System of your instance to the latest version, the migration of your data using rsync is recommended.

Power off your instance

1 . Connect to your Scaleway Console and click on Instances in the side menu.

2 . Select the C2 to snapshot and click on it’s name to enter the Instance Overview page, then click on the Power off button at the top of the page.

Note: Depending on the amount of volumes connected to your instance, powering it down may take some time. Be careful when powering down C2-L instances. These instances have a directly connected DSSD volume which will be deleted when powering down the instance. Make sure to backup your data prior to shutting down the instance.

Creating a Snapshot

1 . Once the C2 instance has been powered off, click on Attached Volumes in the menu bar.

2 . The list of attached volumes displays. Choose the volume to snapshot and click on , then Snapshot:

3 . Enter a friendly name for the snapshot and click on Take snapshot:

4 . On the instances overview page, click Detach to detach the flexible public IP address of the instance. It will be attached to the new instance in a later step.

Creating a new Instance

1 . Click on the Instances tab, then + Create an instance to launch the instances creation wizard.

2 . Click on the My Snapshots tab in the Choose an Image section of the wizard. Choose the previously created snapshot from the list:

3 . Choose the instance for your needs from the list of available General Purpose and Development instances. Make sure that the new instance has at least as much local storage, as the size of your Snapshot (For example: If you have a 50GB snapshot, the new instance must provide at least 50GB of available local storage).

4 . Configure the Local Storage of the instance. If the instance provides more storage than the size of the snapshot, configure another volume to use all available local storage:

5 . Click on +Advanced Options and choose the flexible IP for the instance from the drop down list.

6 . Configure a Bootscript in the advanced settings, a bootscript uses a pre-built kernel to boot from the network. It is a requirement for C2-based volumes. Choose the same bootscript as on your current instance from the drop-down list. To boot on the latest stable kernel select the x86_64 mainline latest boot script.

7 . Deploy the new instance by clicking on Create a new Instance at the bottom of the page.

8 . Once everything is up and runnning again, delete the C2 instance by clicking on the DELETE button on the instances overview page.

Solution 2 - Migrating Using rsync and Block Storage Volumes

It is possible to migrate data stored on an intance using the open-source tool rsync and Scaleway Block Storage Volumes, which provide highly available and redundant storage. An ideal solution for C2 instances with large amounts of data, the migration of ARM64 instances or for the migration to Bare Metal Servers.

If you want to migrate to a Bare Metal Server, or if the default storage capacity of the Virtual Instance is sufficient for your use case, you can launch the data migration directly on the physical disks of the server. You can skip the procedure for formatting and mounting of the Block Storage and move forward directly to the Creating a temporary SSH Key section.

Creating a Virtual Instance

1 . Connect to your Scaleway Console and click on Instances in the side menu.

2 . Click on +Create an instance to launch the instances creation wizard.

3 . Configure the instance towards your requirements from the available offers.

4 . In the Add Volumes section of the creation wizard configure a Block Storage volume to store your transferred data on.

Note: You may choose the size of the volume on your requirements. Make sure that the volume has at least the size of the amount of data to transfer.

5 . Click Add Block Storage to add the volume to the instances’ configuration. The new Block displays in the volumes list. Name the volume to identify it more easily. To modify the size of the volume use the + and - buttons.

6 . Click on Create a new instance at the bottom of the page to launch the creation of the new instance. It will be deployed from a fresh operating system image, supporting local boot.

Mounting the Block Volume

1 . Connect to the newly created instance with ssh:

ssh root@<your_instance_ip>

2 . Use the lsblk command to check that the volume is available:

# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda       8:0    0 23.3G  0 disk
vda     252:0    0 18.6G  0 disk
├─vda1  252:1    0 18.5G  0 part /
└─vda15 252:15   0  100M  0 part /boot/efi

Your can see your block volume named sda with the chosen storage size.

Note: The Scaleway ecosystem uses GiB to define storage sizes and not GB as the default on linux.

3 . Format the block storage volume with a ext4 filesystem.

# mkfs.ext4 /dev/sda

4 . Verify that the file system has been created successfully by running the lsblk -f command.

# lsblk -f
NAME    FSTYPE LABEL UUID                                 MOUNTPOINT
sda     ext4         925c2c17-ca8c-493a-a9cd-b475d87fd276
vda
├─vda1  ext4   root  8509fe04-d91c-410a-9087-c5d34537d3ae /
└─vda15 vfat         B3E7-7040                            /boot/efi

Check if the FSTYPE field matches ext4 for the block volume.

5 . Create the mount point for the volume. In this tutorial, is will be /mnt/data.

# mkdir /mnt/data

6 . Then, mount it. It is recommended to use the defaults option:

# mount -o defaults /dev/sda /mnt/data

If you want to see all available options, you can run man mount on your instance.

7 . Make sure your file system is properly mounted by running the lsblk command.

# lsblk
NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda       8:0    0 23.3G  0 disk /mnt/data
vda     252:0    0 18.6G  0 disk
├─vda1  252:1    0 18.5G  0 part /
└─vda15 252:15   0  100M  0 part /boot/efi

Check the MOUNTPOINT field.

8 . To mount the volume automatically during system boot, add this line to the /etc/fstab file of the instance:

echo '/dev/sda /mnt/data ext4 defaults 0 0' >> /etc/fstab

Creating a temporary SSH Key

1 . Connect to the existing C2 server with ssh:

ssh root@<your_c2_server_ip>

2 . Create a new temporary SSH key for the data migration:

# ssh-keygen -t rsa -b 2048 -f rsync-migration-key

3 . Display the content of the public SSH key and copy it into the clipboard:

# cat /root/rsync-migration-key.pub

4 . Connect to the newly created virtual instance and add the public SSH key of the C2 server to the file /root/.ssh/authorized_keys.
Open a text editor like nano paste and paste the content of the clipboard at the end of the file:

# nano /root/.ssh/authorized_keys

Save the file, exit the text loader and logout of the destination instance before continuing.

Migrating Data Using rsync

As the newly created instance with its attached Block Storage volume is up and running now, launch the data migration.

Note: Keep in mind, if you are migrating from ARM64 instances you can migrate your data (websites, photos, documents, …). Binary files should be re-installed directly on the new instance to make sure they are built for x86_64 architecture.

1 . A very easy and common way to do so is using the rsync tool. Connect to the C2 or ARM64 instance using ssh and install the tool using the apt packet manager:

apt update && apt install rsync

2 . Once rsync is installed, it’s quite easy to use. Just run the command:

# rsync -av -e "ssh -i /root/rsync-migration-key" source_directory/ <virtual_instance_ip_address>:/mnt/data/

This way rsync will copy recursively all files from source_directory/ to the new virtual instance and preserve all files permissions.

To to exclude some specific files from being transferred, the --exclude option can be used. For example, to ignore all .pyc and .o files in the source directory, the command line becomes:

# rsync -av -e "ssh -i /root/rsync-migration-key" \
  --exclude='*.o' \
  --exclude='*.pyc'  \
  source_directory/ <virtual_instance_ip_address>:/mnt/data/

3 . Once the data is migrated, remove the temporary SSH key from the C2 or ARM64 instance:

# rm /root/rync-migration-key*

Moving the Flexible IP

1 . Once the transfer of your data is completed, detach the flexible IP of the C2 or ARM64 instance from the instances overview page:

2 . Detach the public flexible IP from the newly created instance. Then click Attach and choose the flexible IP of the C2 or ARM64 instance from the drop down list:

3 . Click Attach IP to confirm. The newly created instance is now reachable using the flexible IP address of the previous C2 or ARM64 instance.

4 . Once everything is up and running, you may delete old compute instance by clicking the DELETE button from the instances overview page.

Note: Currently it is not yet possible to move flexible IPs between Virtual Instances and Bare Metal Servers. In case of migration to a Bare Metal Server, your services will be available on the IP of the dedicated server. You might need to reconfigure them, if required.

Discover a New Cloud Experience

Deploy SSD Cloud Servers in seconds.