Setting the preferred Project does not automatically give the API key bearer permissions for Object Storage in this Project. Ensure that the user/application is either the Owner of the Organization, or has a policy giving them appropriate permissions for Object Storage in this Project. Note that the application of Object Storage permissions can take up to 5 minutes after creating a new rule or policy.
Using IAM API keys with Object Storage
You can carry out actions on Scaleway Object Storage resources either via the Scaleway console, or via a third-party API or CLI, such as the AWS CLI, MinIOClient or Rclone.
While the Scaleway console gives you the option to specify the Scaleway Project to carry out your Object Storage actions in, this option is not available via third-party API/CLI tools. These tools are based on a standard S3 programming interface, which does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI.
In this document, we explain the concept of preferred Projects for Object Storage, explain how to configure your IAM API key for this, and give some code examples for overriding the preferred Project when making an API call.
What is a Project?
You can create multiple Projects within an Organization, to enable you to group your resources. Each Organization has at least one default Project to which all resources you create (Instances, Kubernetes Kapsules, Object Storage buckets, etc.) are added if no other Projects are created.
When you create new Projects, you can choose the Project in which you want to add or manage resources. However, Object Storage has its own limitations which make it harder to manipulate Projects, specifically via an API. When carrying out actions on Scaleway Object Storage resources via the Scaleway console, the target Project can be still selected from the Project dashboard or drop-down menu.
API keys
You need an API key if you want to carry out actions on Scaleway products and resources via the Scaleway API or other Scaleway developer tools like the CLI and Terraform, or any third-party API/CLI compatible with Scaleway products. An API key is not necessary if you only use the console to create and manage your Scaleway resources.
When you generate an API key with IAM, the key is associated with a specific IAM user or IAM application. The API key inherits the permissions of its bearer (the user or application it is associated with). The user/application may have permissions on one or several Scaleway Projects, accorded to them via policies.
The impact of preferred Projects
When you perform an action on Scaleway Object Storage resources using a third-party API or CLI, such as the AWS CLI, MinIOClient or Rclone, you are using tools based on a standard S3 programming interface. This standard interface does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI. The preferred Project is specified when creating the API key (or can be edited at a later date).
When using the S3 CLI:
- An action of listing the buckets (
aws s3 ls
) will list the buckets of the preferred Project - An action of creating a bucket (
aws s3 mb
) will create a new bucket inside the preferred Project - An action of moving an object from a bucket to another (
aws s3 mv source destination
) will only work if the source bucket and the destination buckets are in the preferred Project for an API key
Therefore, whenever you perform an action on Object Storage via the API, check that:
- You are using the correct API key to perform the call
- The API key you use has the adequate permissions on Object Storage on this Project
- The preferred Project for the API key is where you want to perform the action
How to create an API key via the Scaleway console
To create an API key via the Scaleway console, follow the steps detailed in how to create an API key.
During creation, you are asked to select a preferred Project for the API key. A list of your current Projects displays. Choose the required Project for this API key. In the example below, the default project has been chosen:
While creating an API key, you have the option to skip the step of selecting a preferred Project. Note that if you select this option, the Project that you are currently navigating in the console (i.e. the one that is selected in your Project dashboard) will be automatically selected as the preferred Project for Object Storage. This can still be edited later.
It is also essential to make sure the API key has permissions to carry out Object Storage actions within this Project. Ensure that a suitable policy is attached to the IAM user/application associated with this API key.
How to edit the preferred Project of an API key via the Scaleway console
From the API key list, click on «See more Icon» icon to the right of the API key you want to edit, and select Edit* from the menu. A pop-up displays, and you can choose a new preferred project for Object Storage for this API key
How to create and edit API keys via the API
API keys can also be created and edited from the Scaleway API, including the configuration of the preferred Project. See our dedicated developers documentation for more details.
Overriding the preferred Project when making a call
It is possible to override the API key’s preferred Project when making an API call. To do this, you need the Project ID for the target Project where you want to perform the action.
When passing the API access key in the call, simply add the @
symbol to the end of the key, followed by the target Project ID.
For example, imagine we have:
- API access key
SCW2DVV7CZHD8S68YETV
- Target Project with an ID of
50a526f2-c070-4dca-8f51-fe611a17abb2
Passing the API access key SCW2DVV7CZHD8S68YETV@50a526f2-c070-4dca-8f51-fe611a17abb2
will perform actions on the targeted Project even if the preferred Project is different. You must still make sure the API key has the permissions on the targeted Project.
Example
Part 1: Carrying out Object Storage operations via the AWS CLI
In the first part of this example, we create an IAM application, create an API key for that application, and explore the effects that permissions and preferred Project has on our API calls for Object Storage.
Before you start
To complete the actions presented below, you must have:
- A Scaleway account logged into the console
- Owner status or IAM permissions allowing you to create an IAM application.
-
Create a new IAM application called
my-test-app
without attaching any policies. -
Create an API key for the
my-test-app
application. Select the preferred Project for the API key asdefault
. -
Install the AWS CLI on your machine, using the API key you created in step 2 to complete the fields of the
.aws/credentials
file. -
Run the following command to list all buckets:
aws s3 lsThe following error displays:
An error occurred (AccessDenied) when calling the ListBuckets operation: Permission denied.This is because the IAM application that bears this API key does not have permissions to view or create Object Storage resources.
-
Create a policy with the name
object-storage-full-access
and the IAM application you created in step 1 as the principal. When adding rules, create a rule givingAccess to resources
>default
Project, and theStorage
permission setObjectStorageFullAccess
. -
Repeat the command from step 4:
aws s3 lsNow that the application and therefore its API key have appropriate permissions for Object Storage on the
default
Project, a list of this Project’s Object Storage buckets displays. If there are no buckets in the Project, the output is blank. -
Use the following command to create a new bucket called
test-bucket-123
:aws s3 mb s3://test-bucket-123` -
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on the default Project. The bucket you just created with the IAM application’s API key appears in the list of buckets:
Part 2: Overriding the preferred Project
In the second part of this example, we create a new Project, give my-test-app
Object Storage permissions in this Project, and see how we can override the API key’s preferred Project of default
to create a new bucket in my-second-project
.
-
Create a new Project in the Organization, with the name
my-second-project
.NoteYou must be Owner of the Organization, or have the
ProjectManager
,IAMManager
orOrganizationManager
permissions to create a Project. -
From the Project Dashboard, ensure you are in
my-second-project
, and copy the Project ID. -
Open the
~/.aws/credentials
file on your machine, and edit the API access key to add@<Project ID>
at the end, replacing<Project ID>
with the ID you copied in step 10.nano ~/.aws/credentials
aws_access_key_id = SCWVD8M2PZA01RWVNG9AR@a529d1e90-3c37-4a4d-a1d1-62e84f371fd1aws_secret_access_key = 12826d00-5ef6-45e8-a559-xxxxxxxxxSave and exit the file.
-
Edit the
object-storage-full-access
policy you created in part 1 step 5 to add a second rule withAccess to resources
>test-project
as scope, and theStorage
permission setObjectStorageFullAccess
. Alternatively, edit the existing rule to change the scope toAll current projects
. -
Run the following command on your machine to create a new bucket called
bucket-for-second-project
:aws s3 mb s3://bucket-for-second-projectAs we overrode the preferred Project for the API key in step 3, this bucket has been created in the
my-second-project
, rather than thedefault
Project -
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on my-second-project. The bucket you just created with the IAM application’s API key appears in the list of buckets: