Using Object Storage with s3cmd
In this tutorial, you will learn how to use s3cmd as a client for Scaleway Object Storage. s3cmd
is a tool that allows you to create, list and delete buckets from the command line, as well as to download, upload, and delete objects to/from your buckets.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to perform actions in the intended Organization
- A valid API key
Installing s3cmd
You can install s3cmd from the command line using your Linux package manager.
- Run the following command in a terminal to update and upgrade your packages:
apt update && apt upgrade -y
- Run the following command to install s3cmd:
apt install s3cmd
Run the following command to install s3cmd:
brew install s3cmd
Configuring s3cmd
Carry out the following commands from your terminal.
- Create a configuration file named
.s3cfg
in your home directory:cd $HOME touch .s3cfg
- Open the file with a text editor, e.g. nano:
nano .s3cfg
- Copy and paste the following text into the file, then save and close. Make sure you use the
host-base
,host-bucket
, andbucket_location
details of your own bucket (visible in the Bucket settings tab of the console). You will also need to replace<ACCESS_KEY>
and<SECRET_KEY>
with your own API credentials.[default] # Object Storage Region NL-AMS host_base = s3.nl-ams.scw.cloud host_bucket = %(bucket).s3.nl-ams.scw.cloud bucket_location = nl-ams use_https = True # Login credentials access_key = <ACCESS_KEY> secret_key = <SECRET_KEY>
Using multiple configuration files
You can configure s3cmd to use different configuration files on the same computer to manage buckets in multiple regions. Here, we assume that your default configuration file was for a bucket in nl-ams
, and we create a new configuration file for buckets in fr-par
.
Carry out the following steps from the terminal.
- Create a default configuration file as indicated in the previous step.
- Create a new file named
.s3cfg-fr-par
in your home directory.cd $HOME touch .s3cfg-fr-par
- Copy and paste the following text into the file, then save and close. Make sure you replace
<ACCESS_KEY>
and<SECRET_KEY>
with your own API credentials.[default] signature = s3v4 # Object Storage Region FR-PAR bucket_location = fr-par host_base = https://s3.fr-par.scw.cloud host_bucket = https://s3.fr-par.scw.cloud # Login credentials access_key = <ACCESS_KEY> secret_key = <SECRET_KEY>
- Use s3cmd by specifying the new configuration file in the command:
s3cmd ls -c ~/.s3cfg-fr-par
Using Object Storage with s3cmd
Creating a new bucket
Enter the following command to create a bucket with s3cmd. The bucket name must contain only alphanumeric and lowercase characters. Here, we call it mynewbucket
:
s3cmd mb s3://mynewbucket
An output similar to the following displays:
Bucket 's3://mynewbucket/' created
Uploading objects into the bucket
Enter the following command to upload files into a bucket. Here, we upload the files movie1.avi
and photo1.jpg
into mynewbucket
:
s3cmd put movie1.avi photo1.jpg s3://mynewbucket
An output similar to the following displays:
movie1.avi -> s3://mynewbucket/movie1.avi [1 of 2]
0 of 0 0% in 0s 0.00 B/s done
photo1.jpg -> s3://mynewbucket/photo1.jpg [2 of 2]
0 of 0 0% in 0s 0.00 B/s done
Listing objects inside a bucket
Enter the following command to list files inside a bucket. Here, we list the files in the bucket mynewbucket
:
s3cmd ls s3://mynewbucket
An output similar to the following displays, showing the files inside your bucket:
2014-09-10 08:44 0 s3://mynewbucket/movie1.avi
2014-09-10 08:44 0 s3://mynewbucket/photo1.jpg
Downloading an object from a bucket
Enter the following command to download an object from your bucket to your local machine. Here, we download the object movie1.avi
from the bucket mynewbucket
:
s3cmd get s3://mynewbucket/movie1.avi movie1.avi
An output similar to the following displays:
s3://mynewbucket/movie1.avi -> movie.avi [1 of 1]
0 of 0 0% in 0s 0.00 B/s done
Removing objects from a bucket
Enter the following command to remove an object from your bucket. Here, we remove the object movie1.avi
from the bucket mynewbucket
:
s3cmd del s3://mynewbucket/movie1.avi
An output similar to the following displays:
delete: 's3://mynewbucket/movie1.avi'
Removing a bucket
Enter the following command to remove a bucket. Here, we remove the bucket mynewbucket
:
s3cmd rb s3://mynewbucket
You should see an output similar to the following:
Bucket 's3://mynewbucket/' removed
Configuring CORS
The CORS standard provides browsers with a way to request resources from remote domains when they have permission. This lets you configure how your buckets can be accessed from browser-based applications and websites.
Setting a CORS configuration on a bucket
- Create a CORS configuration file and open it in a text editor:
nano cors.xml
- Put your configuration in the file, save it, and exit the editor:
<CORSConfiguration> <CORSRule> <AllowedOrigin>http://www.example.com</AllowedOrigin> <AllowedMethod>PUT</AllowedMethod> <AllowedMethod>POST</AllowedMethod> <AllowedMethod>DELETE</AllowedMethod> <AllowedHeader>*</AllowedHeader> <MaxAgeSeconds>3000</MaxAgeSeconds> <ExposeHeader>x-server-side-encryption</ExposeHeader> </CORSRule> </CORSConfiguration>
- Set the information on the bucket. Make sure to replace
bucketname
with the name of your bucket:s3cmd setcors cors.xml s3://bucketname
Deleting the CORS configuration of a bucket
Run the following command to delete the CORS configuration of a bucket. Replace bucketname
with the name of your bucket:
s3cmd delcors s3://bucketname
Going further
For more information about the different s3cmd commands, refer to the official documentation, or run the following command in a terminal:
s3cmd --help
Visit our Help Center and find the answers to your most frequent questions.
Visit Help Center