Using IAM API keys with Object Storage
While the Scaleway console gives you the option to specify the Scaleway Project to carry out your Object Storage actions in, this option is not available via third-party API/CLI tools. These tools are based on a standard S3 programming interface, which does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI.
In this document, we explain the concept of preferred Projects for Object Storage, explain how to configure your IAM API key for this, and give some code examples for overriding the preferred Project when making an API call.
You can create multiple Projects within an Organization, to enable you to group your resources. Each Organization has at least one Project, called
default, to which all resources you create (Instances, Kubernetes Kapsules, Object Storage buckets etc) are added if no other Projects are created.
When you create new Projects, you can choose which Project to add resources to, or manage resources in. However, Object Storage has its own limitations which make it harder to manipulate Projects, specifically via an API. When carrying out actions on Scaleway Object Storage resources via the Scaleway console, the target Project can be still selected from the Project dashboard or drop-down menu.
You need an API key if you want to carry out actions on Scaleway products and resources via the Scaleway API or other Scaleway developer tools like the CLI and Terraform, or any third-party API/CLI compatible with Scaleway products. An API key is not necessary if you only use the console to create and manage your Scaleway resources.
When you generate an API key with IAM, the key is associated with a specific IAM user or IAM application. The API key inherits the permissions of its bearer (the user or application it is associated with). The user/application may have permissions on one or several Scaleway Projects, accorded to them via policies.
When you perform an action on Scaleway Object Storage resources using a third-party API or CLI, such as the AWS CLI, MinIOClient or Rclone, you are using tools based on a standard S3 programming interface. This standard interface does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI. The preferred Project is specified when creating the API key (or can be edited at a later date).
Setting the preferred Project does not automatically give the API key bearer permissions for Object Storage in this Project. Ensure that the user/application is either the Owner of the Organization, or has a policy giving them appropriate permissions for Object Storage in this Project. Note that the application of Object Storage permissions can take up to 5 minutes after creating a new rule or policy.
When using the S3 CLI:
- An action of listing the buckets (
aws s3 ls) will list the buckets of the preferred Project
- An action of creating a bucket (
aws s3 mb) will create a new bucket inside the preferred Project
- An action of moving an object from a bucket to another (
aws s3 mv source destination) will only work if the source bucket and the destination buckets are in the preferred Project for an API key
Therefore, whenever you perform an action on Object Storage via the API, check that:
- You are using the correct API key to perform the call
- The API key you use has the adequate permissions on Object Storage on this Project
- The preferred Project for the API key is where you want to perform the action
To create an API key via the Scaleway console, follow the steps detailed in how to create an API key.
During creation, you are asked to select a preferred Project for the API key. A list of your current Projects displays. Choose the required Project for this API key. In the example below, the default project has been chosen:
While creating an API key, you have the option to skip the step of selecting a preferred Project. Note that if you select this option, the Project that you are currently navigating in the console (i.e. the one that is selected in your Project dashboard) will be automatically selected as the preferred Project for Object Storage. This can still be edited later.
It is also essential to make sure the API key has permissions to carry out Object Storage actions within this Project. Ensure that a suitable policy is attached to the IAM user/application associated with this API key.
From the API key list, click on «See more Icon» icon to the right of the API key you want to edit, and select Edit* from the menu. A pop-up displays, and you can choose a new preferred project for Object Storage for this API key
API keys can also be created and edited from the Scaleway API, including the configuration of the preferred Project. See our dedicated developers documentation for more details.
It is possible to override the API key’s preferred Project when making an API call. To do this, you need the Project ID for the target Project where you want to perform the action.
When passing the API access key in the call, simply add the
@ symbol to the end of the key, followed by the target Project ID.
For example, imagine we have:
- API access key
- Target Project with an ID of
Passing the API access key
SCW2DVV7CZHD8S68YETV@50a526f2-c070-4dca-8f51-fe611a17abb2 will perform actions on the targeted Project even if the preferred Project is different. You must still make sure the API key has the permissions on the targeted Project.
In the first part of this example, we create an IAM application, create an API key for that application, and explore the effects that permissions and preferred Project has on our API calls for Object Storage.
Create a new IAM application called
my-test-appwithout attaching any policies.
Create an API key for the
my-test-appapplication. Select the preferred Project for the API key as
Install the AWS CLI on your machine, using the API key you created in step 2 to complete the fields of the
Run the following command to list all buckets:aws s3 ls
The following error displays:An error occurred (AccessDenied) when calling the ListBuckets operation: Permission denied.
This is because the IAM application that bears this API key does not have permissions to view or create Object Storage resources.
Create a policy with the name
object-storage-full-accessand the IAM application you created in step 1 as the principal. When adding rules, create a rule giving
Access to resources>
defaultProject, and the
Repeat the command from step 4:aws s3 ls
Now that the application and therefore its API key have appropriate permissions for Object Storage on the
defaultProject, a list of this Project’s Object Storage buckets displays. If there are no buckets in the Project, the output is blank.
Use the following command to create a new bucket called
test-bucket-123:aws s3 mb s3://test-bucket-123`
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on the default Project. The bucket you just created with the IAM application’s API key appears in the list of buckets:
In the second part of this example, we create a new Project, give
my-test-app Object Storage permissions in this Project, and see how we can override the API key’s preferred Project of
default to create a new bucket in
Create a new Project in the Organization, with the name
You must be Owner of the Organization, or have the
OrganizationManagerpermissions to create a Project.
From the Project Dashboard, ensure you are in
my-second-project, and copy the Project ID.
~/.aws/credentialsfile on your machine, and edit the API access key to add
@<Project ID>at the end, replacing
<Project ID>with the ID you copied in step 10.
nano ~/.aws/credentialsaws_access_key_id = SCWVD8M2PZA01RWVNG9AR@a529d1e90-3c37-4a4d-a1d1-62e84f371fd1aws_secret_access_key = 12826d00-5ef6-45e8-a559-xxxxxxxxx
Save and exit the file.
object-storage-full-accesspolicy you created in part 1 step 5 to add a second rule with
Access to resources>
test-projectas scope, and the
ObjectStorageFullAccess. Alternatively, edit the existing rule to change the scope to
All current projects.
Run the following command on your machine to create a new bucket called
bucket-for-second-project:aws s3 mb s3://bucket-for-second-project
As we overrode the preferred Project for the API key in step 3, this bucket has been created in the
my-second-project, rather than the
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on my-second-project. The bucket you just created with the IAM application’s API key appears in the list of buckets: