Learn how to use s3cmd with Object Storage by following this tutorial.
Requirements
- You have an account and are logged into console.scaleway.com
- You have generated your API Key
Object Storage allows you to store any kind of objects (documents, images, videos, etc.). You can retrieve them anytime and from anywhere.
For instance, you can store images and they will be accessible using HTTP. You can use the control panel to manage your storage. As the our API is S3 compatible, a lot of tools exist to interact with the object storage.
This tutorial details how to use the Object Storage with s3cmd.
There are four steps to configure and use s3cmd:
To retrieve your credentials, refer to S3 credentials.
A simple solution to begin with Object Storage is to use the S3cmd tool, which is a client for S3 Object Storage.
S3cmd allows you to create, list and delete buckets, download, upload, and delete objects inside the Object Storage. Download S3cmd S3cmd on github.
If you use Linux or OS X, your package manager probably has a package for s3cmd.
Important: Make sure you have at least the version 2.0 (
s3cmd --version). Upgrades3cmdif it is not the case. Previous versions might not work properly.
1 . In a terminal, create a file named .s3cfg in your home directory.
2 . Replace access_key = <ACCESS_KEY TO INSERT HERE> and secret_key = <SECRET_KEY TO INSERT HERE> with your credentials.
[default]
# Object Storage Region NL-AMS
host_base = s3.nl-ams.scw.cloud
host_bucket = %(bucket)s.s3.nl-ams.scw.cloud
bucket_location = nl-ams
use_https = True
# Bucket Website feature configuration (https://www.scaleway.com/en/docs/s3-bucket-website-console/)
website_endpoint = https://%(bucket)s.s3-website.%(location)s.scw.cloud/
website_index = index.html # Website index filename
website_error = error.html # Website error filename
# Login credentials
access_key = <ACCESS_KEY TO INSERT HERE>
secret_key = <SECRET_KEY TO INSERT HERE>Alternatively you can also use the configuration file generator by running s3cmd --configure to generate a configuration file.
It is possible to configure s3cmd to use different configuration files on the same computer to manage buckets in multiple regions.
1 . Create a default configuration file as indicated in the previous step.
2 . In a terminal, create a new file named .s3cfg-fr-par in your home directory.
3 . Replace access_key = <ACCESS_KEY TO INSERT HERE> and secret_key = <SECRET_KEY TO INSERT HERE> with your credentials.
[default]
signature = s3v4
# Object Storage Region FR-PAR
bucket_location = fr-par
host_base = https://s3.fr-par.scw.cloud
host_bucket = https://s3.fr-par.scw.cloud
# Login credentials
access_key = <ACCESS_KEY TO INSERT HERE>
secret_key = <SECRET_KEY TO INSERT HERE>4 . Use s3cmd by specifying the new configuration file in the command:
s3cmd ls -c ~/.s3cfg-fr-par
The following command creates a bucket to store objects with S3cmd. A bucket name must contain only alphanumeric and lowercase characters.
john@localhost:~$ s3cmd mb s3://mynewbucket
Bucket 's3://mynewbucket/' createdUploading Objects into the BucketThe following command uploads movie1.avi and photo1.jpg into the bucket mynewbucket:
john@localhost:~$ s3cmd put movie1.avi photo1.jpg s3://mynewbucket
movie1.avi -> s3://mynewbucket/movie1.avi [1 of 2]
0 of 0 0% in 0s 0.00 B/s done
photo1.jpg -> s3://mynewbucket/photo1.jpg [2 of 2]
0 of 0 0% in 0s 0.00 B/s doneImportant: When uploading files from a Scaleway Compute Instance an error message may appear. Your file will be uploaded and you can ignore this message:
ERROR: Cannot retrieve any response status before encountering an EPIPE or ECONNRESET exception WARNING: Upload failed: /bucket_name?partNumber=1&uploadId=ZWY5ZWRhNzItMjE3NS00ZTY4LWFkOTMtYTFhODRkYTRkNzdl ([Errno 32] Broken pipe)We are currently working to avoid this behaviour.
The following command lists the objects inside a bucket:
john@localhost:~$ s3cmd ls s3://mynewbucket
2014-09-10 08:44 0 s3://mynewbucket/movie1.avi
2014-09-10 08:44 0 s3://mynewbucket/photo1.jpgThe following command downloads the object movie1.avi locally:
john@localhost:~$ s3cmd get s3://mynewbucket/movie1.avi movie1.avi
s3://mynewbucket/movie1.avi -> movie.avi [1 of 1]
0 of 0 0% in 0s 0.00 B/s doneThe following command removes the object movie1.avi from the bucket mynewbucket:
john@localhost:~$ s3cmd del s3://mynewbucket/movie1.avi
File s3://mynewbucket/movie1.avi deletedThe following command removes the bucket mynewbucket:
john@localhost:~$ s3cmd rb s3://mynewbucket
Bucket 's3://mynewbucket/' removedThe Bucket Website feature allows you to host a static website in your Object Storage bucket.
Create a website from a bucket:
s3cmd ws-create s3://mybucket
Retrieve information about a website:
s3cmd ws-info s3://mybucket
Delete a website from a bucket:
s3cmd ws-delete s3://mybucket
1 . Create a CORS configuration file and open it in a text editor:
nano cors.xml
2 . Put your configuration in the file, save it and exit the editor:
<CORSConfiguration>
<CORSRule>
<AllowedOrigin>http://www.example.com</AllowedOrigin>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<AllowedHeader>*</AllowedHeader>
<MaxAgeSeconds>3000</MaxAgeSeconds>
<ExposeHeader>x-server-side-encryption</ExposeHeader>
</CORSRule>
</CORSConfiguration>
3 . Set the information on the bucket:
s3cmd setcors cors.xml s3://bucketname
1 . To delete the CORS configuration of a bucket, run the following command:
s3cmd delcors s3://bucketname
Important: It is currently not possible to get the CORS configuration of a bucket with
s3cmd. It is recommended to use aws-cli to get the CORS settings of a bucket.
For more information about the different s3cmd commands, refer to the official documentation by running s3cmd --help.