Help


Documentation & Tutorials

Object Storage

Object Storage Overview

Requirements

Scaleway Object Storage is an Object Storage service based on the S3 protocol. It allows you to store any kind of objects (documents, images, videos, etc.).

Object Storage service is a cloud product which means that your objects are always available from anywhere in the world. For instance, a typical use case for Object Storage would be to store images in a public bucket to make them broadly available via the HTTP protocol.

You can manage your storage directly from the Scaleway Console. When on the control panel, you can easily upload, download and visualize the objects in your buckets. In addition, many existing libraries or CLI clients can be integrated into your application or scripts.

Core Concepts

Object Storage: Object storage, contrary to previous storage methods (block devices, file-systems…), essentially bundles the data itself along with metadata tags and a unique identifier (instead of a file name and a file path). Object storage organizes information into buckets of flexible sizes, referred to as objects.These unique identifiers are arranged in a flat address space which makes it easier to locate and retrieve your data across regions. This flat address space also helps with scalability.

Bucket: A bucket is a groups of objects sharing a common denominator. It can contain as many objects as the you want. However the bucket might slowdown with a large number of objects (>500 000).

S3: S3 is the de facto object storage protocol, it was created by Amazon for its own object storage service, and is the most used in the world. Object Storage officially supports a subset of S3. The list of supported features are described in the Scaleway FAQ.

Signature V2, Signature V4: When you send HTTP requests to AWS, you sign the requests so that AWS can identify who sent them. You sign requests with your AWS access key, which consists of an access key ID and secret access key. The two main s3 protocols for authentication are Signature v2 and Signature v4. Signature v4 is more recent and is the only supported version. Usually, the signature should not be an issue for you as it is computed by the client software or library, not manually.

Region and Endpoint: Just like many Scaleway services, Object Storage will be available in only some regions (AMS-1 to begin with). To put a bucket in one region we need to target one endpoint. For example for the beta:
Region: nl-ams
Endpoint: https://s3.nl-ams.scw.cloud

Supported Open Source Clients

Object Storage Use Cases

Scaleway Object Storage can be used in multiple use cases. To inspire you with common use cases, we came up with different tutorials such as:

More tutorials will follow soon.

Operation Procedures

Most operations can be performed through the S3 protocol via a CLI tool like s3cmd or through Scaleway Console.

To access the S3 services, you need to be in possession of an Access key and Secret key, also called Token-pair. You can generate these credentials from your management console.

Creating a Bucket through the Scaleway Console

1 . Log in to the Scaleway Console

2 . Click Storage from the left side menu. The Storage page lists all your buckets. At first your list is empty as you have not created any bucket yet.

3 . Click Create a Bucket to create a bucket that will store your objects

4 . Name your bucket and validate your bucket creation. A bucket name must:

  • be unique
  • contain only alphanumeric and lowercase characters

Creating a Bucket through the CLI

john@localhost:~$ s3cmd mb s3://mynewbucket

Bucket 's3://mynewbucket/' created

In addition, the full list of the S3 API operations supported, is available here.

Object Storage Limitation

To ensure we provide the best performances, we listed below the current limitations of the feature:

  • Maximum number of bucket: 100
  • Maximum number of objects in a bucket: no specific limit, however, the bucket might slowdown with a large number of objects
  • Maximum size of an object: 1000 parts (parts can be of arbitrary size)
  • Maximum HTTP requests (read and write): 250 requests per second per bucket

In case you need to raise those limitations, contact the Support team.

Troubleshooting and Error Message

The user can encounter the following error message:

_Token invalid: The client token or secret is invalid._

We suggest to check the user’s configuration. For users using v4 signature method, ensure given region is correct (e.g., “nl-ams” for the Amsterdam region).

Object Storage and S3 Compatibility

Refer to Scaleway FAQ for more information regarding S3 compatibility.

A full list of S3 API operations is available here.

Discover a New Cloud Experience

Deploy SSD Cloud Servers in seconds.