Managing an Object Storage Lifecycle using CLI (v2)

Published on 05 August 2021

Scaleway Object Storage is service based on the S3 protocol. It allows you to store different types of objects (documents, images, videos, etc.) and to distribute them instantly, anywhere in the world. You can create and manage your Object Storage resources from the console, or via the Scaleway Command Line Interface v2 that uses external tools such as rclone, s3cmd and mc.

Scaleway Command Line Interface Overview

The Scaleway Command Line Interface (CLI) allows you to pilot your Scaleway infrastructure directly from your terminal, providing a faster way to administer and monitor your resources. Scaleway CLI is easy to set up and is an essential tool for operating efficiently in your cloud environment.

Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

you are the Owner of the Scaleway Organization in which the actions will be carried out, or
you are an IAM user of the Organization, with a policy granting you the necessary permission sets

Requirements:

You have an account and are logged into the Scaleway Console
You have generated your API key
You have downloaded and installed the Scaleway CLI
You have downloaded S3cmd, rclone and mc s3 tools

Configuring the CLI

Open a terminal and type the following command, to configure the Scaleway CLI on your machine:

scw init

If you have already configured the CLI, you will see a message like the following:

Current config is located at /path/to/config.yaml
access_key: <YOUR ACCESS KEY>
secret_key: <YOUR SECRET KEY>
default_organization_id: <YOUR ORGANIZATION ID>
default_project_id: <YOUR PROJECT ID>
default_region: <YOUR DEFAULT REGION, eg fr-par>
default_zone: <YOUR DEFAULT ZONE, eg fr-par-1>
Do you want to override the current config? (Y/n):

Type n and go directly to the Creating a configuration file for the Scaleway CLI v2 step of this tutorial if you do not want to edit your configuration.

If you have not yet configured the CLI, you will be guided through the configuration process via a series of questions:
```
Enter a valid secret-key or an email
```

Enter either the email address for your account (you will then be asked for your password), or your API Secret Key:

To improve this tool we rely on diagnostic and usage data.
Sending such data is optional and can be disabled at any time by running "scw config set send-telemetry=false".
Do you want to send usage statistics and diagnostics? (Y/n):

Type y or n and hit enter.

To fully enjoy Scaleway CLI we recommend you install autocomplete support in your shell.
Do you want to install autocomplete? (Y/n):

Type y or n and hit enter.

To enable autocomplete, scw needs to update your shell configuration.
What type of shell are you using (default: zsh):

Hit enter if the default shell type is already correct for you. Otherwise, enter your shell type and hit enter.

To enable autocomplete we need to append to /Users/yourusername/.zshrc the following lines:
# Scaleway CLI autocomplete initialization.
eval "$(scw autocomplete script shell=zsh)"
Do you want to proceed with these changes? (Y/n):

Type y or n and hit enter. This should complete the configuration process:
```
Initialization completed with success.
```

Creating a configuration file for the Scaleway CLIv2

Important:

Unless you add a region argument to your commands, the configuration file will be set for the default region.

Type the following command in your terminal to create a configuration file with rclone:

scw object config get type=rclone

You will see the following output:

scw object config get type=rclone
# Generated by scaleway-cli command
# Configuration file for rclone https://rclone.org/s3/#scaleway
# Default location: $HOME/.config/rclone/rclone.conf
[scaleway]
type = s3
env_auth = false
endpoint = s3.fr-par.scw.cloud
access_key_id = SCW85PRCA6HD4PDEWS5M
secret_access_key = 8bd5d8ce-d476-4bd5-b97d-2d97e04c3c4d
region = fr-par
location_constraint =
acl = private
force_path_style = false
server_side_encryption =
storage_class =

Type the following command in your terminal to create a configuration file with s3cmd:

scw object config get type=s3cmd

You will see the following output:

scw object config get type=s3cmd
# Generated by scaleway-cli command
# Configuration file for s3cmd https://s3tools.org/s3cmd
# Default location: $HOME/.s3cfg
[default]
access_key = SCW85PRCA6HD4PDEWS5M
bucket_location = fr-par
host_base = s3.fr-par.scw.cloud
host_bucket = %(bucket)s.s3.fr-par.scw.cloud
secret_key = 12345678-1234-1234-4321-123456789123
use_https = True

Type the following command in your terminal to create a configuration file with mc:

scw object config get type=mc

You will see the following output:

scw object config get type=mc
{"version":"9","hosts":{"scaleway":{"url":"https://s3.fr-par.scw.cloud","accessKey":"SCW1111111111111","secretKey":"12345678-1234-1234-4321-123456789123","api":"S3v4"}}}

Note:

For more information about generating a configuration file, type the following command: scw object config get --help in your terminal.

Installing a configuration file for S3 tools (s3cmd, rclone, and mc)

Type the following command in your terminal to install a configuration file for s3cmd:

scw object config install type=s3cmd

You will see an output similar to this:
```
scw object config install type=s3cmd
Configuration file successfully installed at /Users/yourusername/.s3cfg.
```
Type the following command in your terminal to install a configuration file for rclone:

scw object config install type=rclone

You will see an output similar to this:
```
scw object config install type=rclone
Configuration file successfully installed at /Users/yourusername/.config/rclone/rclone.conf.
```
Type the following command in your terminal to install a configuration file for mc:

scw object config install type=mc

You will see an output similar to this:
```
scw object config install type=mc
Configuration file successfully installed at /Users/yourusername/.mc/config.json.
```

Adding an Object Storage bucket

Type the following command in your terminal to create a bucket:

s3cmd mb s3://your-bucket

You will see the the following output:
```
Bucket 's3://your-bucket/' created
```
Type the following command in your terminal to add an object to your storage:

s3cmd put file /Users/yourusername/.config/scw/config.yaml s3://your-bucket/your-object

You will see the following output:
```
upload: '/Users/yourusername/.config/scw/config.yaml' -> 's3://your-bucket/your-object'  [1 of 1]
3259 of 3259   100% in    0s     8.47 KB/s  done
```

Removing an object from a bucket

Type in the following command in your terminal:

s3cmd delete: s3://your-bucket/your-object

You will see the following output

delete: 's3://your-bucket/your-object'

Note:

For more information about removing objects type the following command: s3cmd --help in your terminal.

Deleting an Object Storage bucket

Important:

This will result in the deletion of every object stored inside your bucket. Your bucket name will no longer be yours and other users will be able to use it.

Type the following command in your terminal to display your buckets:

s3cmd ls

You will see an output similar to the following:
```
s3cmd ls
2021-09-01 15:56  s3://name-of-your-bucket
```
Type the following command in your terminal to delete a bucket:

s3cmd rb s3://your-bucket-name

You will see the following output:
```
Bucket 's3://name-of-your-bucket/' removed
```

Tip:

For more information about the s3 tools used in this documentation, refer to the official rclone, s3cmd, and mc documentations.