Scaleway Block Storage

Block Storage Overview

Scaleway Block Storage provides network attached storage that can be plugged-in/out of cloud instances like a virtual hard-drive. From a user’s point-of-view, the block device behaves like a regular disk.

When a block device is being connected to a cloud instance, it is being detected as a raw, unformatted disk by the operating system. It is possible to create any file system on it (ext4, XFS, NTFS…) and use the device to store data like on any other disk.

In the background, a block device is managed by the cluster as a collection of smaller pieces (called chunks or simply blocks, hence its name). Each of these chunks can be stored across a storage cluster of several machines and under a unique address. In particular, in the case of a cloud block storage, those chunks are replicated to avoid data loss in the event of a storage medium failure.

When a particular file is being requested from the operating system of the cloud instance, it sends a request to the block device holding the file. This request is then translated by the block storage system which will retrieve the chunks of the file, re-assemble them and send the file to the operating system like a classic drive would do.

As this process is much faster than searching for a file on a traditional hard-disk it makes block storage ideal for latency-critical applications such as storage of virtual machines and transactional databases. It provides also an extra layer of redundancy for business-critical applications, as data is stored across multiple physical drives and nodes. In case of a hardware failure on one of the disks, the missing chunks can easily be recovered from other disks in the cluster.

Block Storage devices are also independent from the local storage of cloud instances. In addition, the fact that they are accessed over a network connection makes it easy to detach them from one instance and re-attached (or not) to another instance within the same availability zone.

Requirements

Core Concepts

Block Device: A Block Device is a storage volume on a network connected storage system that is exposed to the guest operating system as if it would be a physical disk.

Storage Area Networks (SANs): A Storage Area Network, or abbreviated SAN, consists of interconnected machines, network infrastructure and storage devices designed for performance and high-availability. Unlike a Network Attached Storage (NAS) which stores all data on file level, the SAN stores all data on block level, making it a perfect solution for business critical applications and input/output intense operations like relational databases. The SAN is a network connected solution, that operates independently from the local hypervizor hosting the virtual instance, the storage capacity of the block devices on the SAN can be tailored towards your personal requirements.

Local Volume: The local volume of an instance is an all SSD based storage solution, using a RAID array for redundancy and performance, that is being hosted on the local hypervizor. On Scaleway Virtual Cloud Instances the size of the Local Volume is fixed and depending on the offer. It is always possible to increase the storage of a Virtual Cloud Instance by adding additional block volumes to an instance.

Operation Procedures

Block Storage volumes can be created and managed either from the Scaleway console or by using the REST-API.

Once created, the block device can be mounted and formatted directly in the operating system of the virtual cloud instance.

Creating a Block Storage Volume Through the Scaleway Console

1 . Log in to the Scaleway Console

2 . Click Instances from the left side menu. The Instances section displays.

3 . Click Volumes to display a list of all your current volumes. If you do not have created any volume yet, the list will be empty.

3 . Click Create a Volume to create a volume that can be attached to any of your virtual cloud instances.

4 . Choose a region for the volume. Important: The volume must be in the same region as the virtual cloud instance you want it connect to.

5 . Configure the volume by choosing Block Storage. Enter the volume name and select a size.

6 . Click Create a new volume to confirm the creation of the volume.

Creating a Block Storage Volume Through the API

1 . Create a block volume from the API by launching the following command:

$ curl -q \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json' \
	-X POST \
	-d '{"name": "block-volume-101", "organization": "'$SCW_ORGANIZATION'", "size": 10000000000, "volume_type": "b_ssd"}' \
	https://api.scaleway.com/instance/v1/zones/fr-par-1/volumes

2 . To retrieve the current status of the block volume, query the volume endpoint:

$ curl -q \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json'  \
	https://api.scaleway.com/instance/v1/zones/fr-par-1/volumes/b3a42fb1-e85c-46e9-b0a6-9adb62278295
{
  "volume": {
    "size": 10000000000,
    "state": "available",
    "name": "block-volume101",
    "modification_date": "2019-09-03T09:57:53.742130+00:00",
...
    "id": "b3a42fb1-e85c-46e9-b0a6-9adb62278295",
    "volume_type": "b_ssd",
    "server": null
  }
}

Attaching a Block Storage Volume Through the Scaleway Console

1 . Log in to the Scaleway Console

2 . Click Instances from the left side menu. The Instances section displays.

3 . Click Volumes to display a list of all your current volumes. The newly created volume displays on top of the list:

Block Volume created

4 . Click Attach to instance to choose the instance to which the volume shall be attached.

5 . Select an instance from the drop-down list and click on Attach this volume to attach it to the selected instance.

Attach Block Volume

6 . Once the volume is attached to the instance, continue by mounting and formatting the volume.

Connecting a Block Storage Volume Through the API

1 . Query the current volumes of the instance:

$ curl -q \
 -H "X-Auth-Token: $SECRET_KEY" \
 -H 'Content-Type: application/json' \
 https://api.scaleway.com/instance/v1/zones/fr-par-1/servers/be3c50af-e8f3-4ff4-90fe-66972f06670d
{
 "server": {
   "allowed_actions": [
...
   "volumes": {
     "0": {
...
       "id": "7fe25caf-0a68-46a7-aeb9-63a278d33e2b",
...
     }
   },
...
}

As displayed, our current volumes are only constituted of the root volume.

2 . Hot-plug the volume created earlier:

$ curl -q \
 -H "X-Auth-Token: $SECRET_KEY" \
 -H 'Content-Type: application/json' \
 -X PATCH \
 -d '{
        "volumes": {
          "0": "7fe25caf-0a68-46a7-aeb9-63a278d33e2b",
          "1": "b3a42fb1-e85c-46e9-b0a6-9adb62278295"
        }
     }' \
 https://api.scaleway.com/instance/v1/zones/fr-par-1/servers/be3c50af-e8f3-4ff4-90fe-66972f06670d
{
 "server": {
...
   "id": "be3c50af-e8f3-4ff4-90fe-66972f06670d",
...
   "name": "scw-blissful-engelbart",
   "protected": false,
   "volumes": {
     "0": {
       "size": 20000000000,
       "state": "available",
...
       "id": "7fe25caf-0a68-46a7-aeb9-63a278d33e2b",
       "volume_type": "l_ssd",
       "server": {
         "id": "be3c50af-e8f3-4ff4-90fe-66972f06670d",
         "name": "scw-blissful-engelbart"
       }
     },
     "1": {
       "size": 10000000000,
       "state": "available",
       "name": "block-volume101",
       "modification_date": "2019-09-03T10:17:40.800839+00:00",
...
       "id": "b3a42fb1-e85c-46e9-b0a6-9adb62278295",
       "volume_type": "b_ssd",
       "server": {
         "id": "be3c50af-e8f3-4ff4-90fe-66972f06670d",
         "name": "scw-blissful-engelbart"
       }
     }
   },
...
 }
}

3 . Use SSH to log into your server and verify that the new disk exists:

$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda       8:0    0 953.7M  0 disk
vda     252:0    0  18.6G  0 disk
├─vda1  252:1    0  18.5G  0 part /
└─vda15 252:15   0   100M  0 part /boot/efi

The block volume is presented inside your instance as /dev/sdX, to help you automate things, a symlink is also present in /dev/disk/by-id/ with the id of the volume in it:

$ ls /dev/disk/by-id/
scsi-0SCW_b_ssd_volume-b3a42fb1-e85c-46e9-b0a6-9adb62278295

If you query information about the block volume from the API, the server_id on which it is plugged-in is displayed

$ curl -q \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json'  \
	https://api.scaleway.com/instance/v1/zones/fr-par-1/volumes/b3a42fb1-e85c-46e9-b0a6-9adb62278295
{
  "volume": {
    "size": 10000000000,
    "state": "available",
    "name": "block-volume101",
...
    "id": "b3a42fb1-e85c-46e9-b0a6-9adb62278295",
    "volume_type": "b_ssd",
    "server": {
      "id": "be3c50af-e8f3-4ff4-90fe-66972f06670d",
      "name": "scw-blissful-engelbart"
    }
  }
}

Detaching a Block Storage Volume Through the Scaleway Console

1 . Log in to the Scaleway Console

2 . Click Instances from the left side menu. The Instances section displays.

3 . Click on the instance having the volume attached that you want to detach.

4 . The instance information displays. Click on Attached Volumes

5 . A list of all volumes attached to the instance displays.

6 . Click on next to the volume you want to detach and click Detach in the menu.

Detach Block Volume

7 . Confirm by clicking Detach this volume in the pop-up window.

Detaching a Block Storage Volume Through the API

To detach a block volume from a running instance, relaunch the API call and omit the volume you want to hot unplug:

$ curl -q \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json' \
	-X PATCH \
	-d '{
	       "volumes": {
	         "0": "7fe25caf-0a68-46a7-aeb9-63a278d33e2b",
	       }
	    }' \
	https://api.scaleway.com/instance/v1/zones/fr-par-1/servers/be3c50af-e8f3-4ff4-90fe-66972f06670d

Deleting a Block Storage Volume Through the Scaleway Console

Once a block volume is detached from an instance, you can delete it from the volumes list in the Scaleway console.

1 . Log in to the Scaleway Console

2 . Click Instances from the left side menu. The Instances section displays.

3 . Click on the Volumes tab to view the list of your volumes.

4 . Click next to the volume that you want to delete. Click Delete to delete the volume.

Delete Block Volume

5 . The pop-up menu displays. Confirm the deletion by entering the word DELETE in the popup. Click on Delete this volume to remove the volume from your account.

Important: All data on the volume will be removed when you delete it. Make sure to have a backup of your data before removing the volume.

Deleting a Block Storage Volume Through the API

After detaching the block volume from the instance, you can also delete it using this api call. Be careful when removing volumes using the API, once deleted your data will be removed:

$ curl -q \
	-X DELETE \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json'  \
	https://api.scaleway.com/instance/v1/zones/fr-par-1/volumes/b3a42fb1-e85c-46e9-b0a6-9adb62278295

Creating an Instance With a Block Storage Volume Through the API Scaleway Console

1 . Log in to the Scaleway Console

2 . Click Instances from the left side menu. The Instances section displays.

3 . Click on Create an Instance and proceed with the steps to create a new instance.

4 . In the Add Volumes click Add block storage and choose the size of the block volume. To create multiple volumes repeat this step as required.

Add Block Volume

5 . Click Create a new instance to launch the instance creation process.

Creating an Instance With a Block Storage Volume Through the API

A virtual instance can be created directly from a block volume. In the example below, we use a DEV1-S instance based on Ubuntu 18.04 with a 10 GB additional block volume :

$ curl -q \
	-X POST \
	-H "X-Auth-Token: $SECRET_KEY" \
	-H 'Content-Type: application/json'  \
	-d '{
	      "name":"blocky",
	      "commercial_type":"DEV1-S",
	      "boot_type":"local",
	      "image":"f974feac-abae-4365-b988-8ec7d1cec10d",
	      "organization":"'$SCW_ORGANIZATION'",
	      "volumes":{
	        "0":{"size":20000000000},
                "1":{
                    "size": 10000000000,
                    "organization":"'$SCW_ORGANIZATION'",
                    "volume_type":"b_ssd",
                    "name":"my-block-volume"
                 }
    }
}
	https://api.scaleway.com/instance/v1/zones/fr-par-1/volumes/b3a42fb1-e85c-46e9-b0a6-9adb62278295

Block Storage Performances

Block Storage offers 5,000 input/output operations per second (IOPS) through powerful SSDs. This guarantees 99.99 % availability to all our customers. However, it is subject to fluctuation during peak load periods. Bear in mind that even-though Block Storage is SSD-based, it remains a distant volume which means that it is expected to be slower than local NVMe attached volume.

Troubleshooting and Error Message

Refer to the Block Storage FAQ or contact our technical assistance for any question you may have.

Discover a New Cloud Experience

Deploy SSD Cloud Servers in seconds.