Help


Community tutorials and documentations

How to use Object Storage with s3cmd

How to use Object Storage with s3cmd

This page shows you how to use Object Storage with s3cmd.

Requirements

  • You have an account and are logged into cloud.scaleway.com
  • You have generate your API Token

Object Storage allows you to store any kind of object (documents, images, videos, etc.). You can retrieve them anytime and from anywhere.

For instance, you can store images and they will be accessible using HTTP. You can use the control panel to manage your storage. As our API is S3 compatible, a lot of existing tools exist to interact with the object storage.

This guide will show you how to use the Object Storage with one of these tools, s3cmd.

There are four steps to configure and use s3cmd:

  • Retrieve your Credentials
  • Download S3cmd
  • Configure S3cmd
  • Object Storage tour with s3cmd

Step 1 - Get your credentials

Before starting, click the “Storage” button in the control panel.

You will land on the Storage page. Then, click on the “S3 Credentials” button.

The credentials required to access the Object Storage are displayed:

  • host_base base url to access the storage service,
  • access_key the access key for the storage,
  • and secret_key the API Token you generated previously.

S3 Crendentials

Step 2 - Download S3cmd

A simple solution to begin with Object Storage is to use the S3cmd tool, which is a client for S3 Object Storage.

S3cmd allows you to create, list and delete buckets, download, upload, and delete objects inside the Object Storage. First, you need to download S3cmd on github here.

If you use Linux or OS X, your package manager probably has a package for s3cmd.

Step 3 - Configure S3cmd

We will configure S3cmd to work with Object Storage. The following is basic configuration example to use Object Storage with S3cmd.

In a terminal, create a file named .s3cfg in your home directory.

Replace access_key = <your_access_key> and secret_key = <your_secret_key> with your credentials.

$cat ~/.s3cfg

[default]
access_key = <your_access_key>
secret_key = <your_secret_key>
default_mime_type = binary/octet-stream
delete_removed = False
dry_run = False
encoding = UTF-8
encrypt = False
follow_symlinks = False
force = False
get_continue = False
guess_mime_type = True
host_base = fr-1.storage.online.net
host_bucket = %(bucket)s.fr-1.storage.online.net
human_readable_sizes = False
list_md5 = False
log_target_prefix =
preserve_attrs = True
progress_meter = True
recursive = False
recv_chunk = 4096
reduced_redundancy = False
send_chunk = 4096
skip_existing = False
socket_timeout = 300
urlencoding_mode = normal
use_https = True
verbosity = WARNING
signature_v2 = True

Step 4 - Object Storage Tour with S3cmd

Create a new bucket

The following command will create a bucket to store objects with S3cmd. A bucket name must contain only alphanumeric and lowercase characters.

john@localhost:~$ s3cmd mb s3://mynewbucket

Bucket 's3://mynewbucket/' created
Upload object into the bucket

The following command will upload movie1.avi and photo1.jpg into the bucket mynewbucket:

john@localhost:~$ s3cmd put movie1.avi photo1.jpg s3://mynewbucket

movie1.avi -> s3://mynewbucket/movie1.avi  [1 of 2]
 0 of 0     0% in    0s     0.00 B/s  done
photo1.jpg -> s3://mynewbucket/photo1.jpg  [2 of 2]
 0 of 0     0% in    0s     0.00 B/s  done

List objects inside a bucket

The following command will list the objects inside a bucket:

john@localhost:~$ s3cmd ls s3://mynewbucket

2014-09-10 08:44         0   s3://mynewbucket/movie1.avi
2014-09-10 08:44         0   s3://mynewbucket/photo1.jpg

Download an object from a bucket

The following command will download the object movie1.avi locally:

john@localhost:~$ s3cmd get s3://mynewbucket/movie1.avi movie1.avi

s3://mynewbucket/movie1.avi -> movie.avi  [1 of 1]
 0 of 0     0% in    0s     0.00 B/s  done

Remove object from a bucket

The following command will delete the object movie1.avi from the bucket mynewbucket:

john@localhost:~$ s3cmd del s3://mynewbucket/movie1.avi

File s3://mynewbucket/movie1.avi deleted

Remove a bucket

The following command will remove the bucket mynewbucket:

john@localhost:~$ s3cmd rb s3://mynewbucket

Bucket 's3://mynewbucket/' removed

Discover a new cloud experience

Deploy SSD cloud servers in seconds.