You can carry out actions on Scaleway Object Storage resources either via the Scaleway console, or via a third-party API or CLI, such as the AWS CLI, MinIOClient or Rclone.
While the Scaleway console gives you the option to specify the Scaleway Project to carry out your Object Storage actions in, this option is not available via third-party API/CLI tools. These tools are based on a standard Amazon S3 programming interface, which does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI.
In this document, we explain the concept of preferred Projects for Object Storage, explain how to configure your IAM API key for this, and give some code examples for overriding the preferred Project when making an API call.
You can create multiple Projects within an Organization, to enable you to group your resources. Each Organization has at least one default Project to which all resources you create (Instances, Kubernetes Kapsules, Object Storage buckets, etc.) are added if no other Projects are created.
When you create new Projects, you can choose the Project in which you want to add or manage resources. However, Object Storage has its own limitations which make it harder to manipulate Projects, specifically via an API. When carrying out actions on Scaleway Object Storage resources via the Scaleway console, the target Project can be still selected from the Project dashboard or drop-down menu.
You need an API key if you want to carry out actions on Scaleway products and resources via the Scaleway API or other Scaleway developer tools like the CLI and Terraform, or any third-party API/CLI compatible with Scaleway products. An API key is not necessary if you only use the console to create and manage your Scaleway resources.
When you generate an API key with IAM, the key is associated with a specific IAM user or IAM application. The API key inherits the permissions of its bearer (the user or application it is associated with). The user/application may have permissions on one or several Scaleway Projects, accorded to them via policies.
When you perform an action on Scaleway Object Storage resources using a third-party API or CLI, such as the AWS CLI, MinIOClient or Rclone, you are using tools based on a standard Amazon S3 programming interface. This standard interface does not accept Project ID as a parameter. Therefore, when you create a Scaleway API key with IAM, you are prompted to specify the API key’s preferred Project for Object Storage. This API key will always use this Project when carrying out Object Storage actions via any API/CLI. The preferred Project is specified when creating the API key (or can be edited at a later date).
Setting the preferred Project does not automatically give the API key bearer permissions for Object Storage in this Project. Ensure that the user/application is either the Owner of the Organization, or has a policy giving them appropriate permissions for Object Storage in this Project. Note that the application of Object Storage permissions can take up to 5 minutes after creating a new rule or policy.
When using the AWS S3 CLI:
aws s3 ls) will list the buckets of the preferred Projectaws s3 mb) will create a new bucket inside the preferred Projectaws s3 mv source destination) will only work if the source bucket and the destination buckets are in the preferred Project for an API keyTherefore, whenever you perform an action on Object Storage via the API, check that:
To create an API key via the Scaleway console, follow the steps detailed in how to create an API key.
During creation, you are asked to select a preferred Project for the API key. A list of your current Projects displays. Choose the required Project for this API key. In the example below, the default project has been chosen:
While creating an API key, you have the option to skip the step of selecting a preferred Project. Note that if you select this option, the Project that you are currently navigating in the console (i.e. the one that is selected in your Project dashboard) will be automatically selected as the preferred Project for Object Storage. This can still be edited later.
It is also essential to make sure the API key has permissions to carry out Object Storage actions within this Project. Ensure that a suitable policy is attached to the IAM user/application associated with this API key.
From the API key list, click on «See more Icon» icon to the right of the API key you want to edit, and select Edit* from the menu. A pop-up displays, and you can choose a new preferred project for Object Storage for this API key
API keys can also be created and edited from the Scaleway API, including the configuration of the preferred Project. See our dedicated developers documentation for more details.
It is possible to override the API key’s preferred Project when making an API call. To do this, you need the Project ID for the target Project where you want to perform the action.
When passing the API access key in the call, simply add the @ symbol to the end of the key, followed by the target Project ID.
For example, imagine we have:
SCW2DVV7CZHD8S68YETV50a526f2-c070-4dca-8f51-fe611a17abb2Passing the API access key SCW2DVV7CZHD8S68YETV@50a526f2-c070-4dca-8f51-fe611a17abb2 will perform actions on the targeted Project even if the preferred Project is different. You must still make sure the API key has the permissions on the targeted Project.
In the first part of this example, we create an IAM application, create an API key for that application, and explore the effects that permissions and preferred Project has on our API calls for Object Storage.
To complete the actions presented below, you must have:
Create a new IAM application called my-test-app without attaching any policies.
Create an API key for the my-test-app application. Select the preferred Project for the API key as default.
Install the AWS CLI on your machine, using the API key you created in step 2 to complete the fields of the .aws/credentials file.
Run the following command to list all buckets:
The following error displays:
This is because the IAM application that bears this API key does not have permissions to view or create Object Storage resources.
Create a policy with the name object-storage-full-access and the IAM application you created in step 1 as the principal. When adding rules, create a rule giving Access to resources > default Project, and the Storage permission set ObjectStorageFullAccess.
Repeat the command from step 4:
Now that the application and therefore its API key have appropriate permissions for Object Storage on the default Project, a list of this Project’s Object Storage buckets displays. If there are no buckets in the Project, the output is blank.
Use the following command to create a new bucket called test-bucket-123:
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on the default Project. The bucket you just created with the IAM application’s API key appears in the list of buckets:
In the second part of this example, we create a new Project, give my-test-app Object Storage permissions in this Project, and see how we can override the API key’s preferred Project of default to create a new bucket in my-second-project.
Create a new Project in the Organization, with the name my-second-project.
You must be Owner of the Organization, or have the ProjectManager, IAMManager or OrganizationManager permissions to create a Project.
From the Project Dashboard, ensure you are in my-second-project, and copy the Project ID.
Open the ~/.aws/credentials file on your machine, and edit the API access key to add @<Project ID> at the end, replacing <Project ID> with the ID you copied in step 10.
nano ~/.aws/credentials
Save and exit the file.
Edit the object-storage-full-access policy you created in part 1 step 5 to add a second rule with Access to resources > test-project as scope, and the Storage permission set ObjectStorageFullAccess. Alternatively, edit the existing rule to change the scope to All current projects.
Run the following command on your machine to create a new bucket called bucket-for-second-project:
As we overrode the preferred Project for the API key in step 3, this bucket has been created in the my-second-project, rather than the default Project
Go to the Scaleway console in your browser, click Object Storage in the side menu, and ensure you are on my-second-project. The bucket you just created with the IAM application’s API key appears in the list of buckets: