Skip to navigationSkip to main contentSkip to footerScaleway Docs

Migrating Scaleway Object Storage data with MinIO client

MinIO
object-storage
s3

The MinIO Client provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff, etc. It can communicate with any Amazon S3-compatible cloud storage provider and can be used to migrate data from one region to another.

Before you start

To complete the actions presented below, you must have:

Installing the MinIO client

  1. Connect to your Instance as root via SSH.
  2. Update the APT package cache and the software already installed on the Instance:
    apt update && apt upgrade -y
  3. Download the MinIO client:
    • For x86 (Intel/AMD):
    curl https://dl.min.io/client/mc/release/linux-amd64/mc \
      --create-dirs \
      -o $HOME/minio-binaries/mc
    
    chmod +x $HOME/minio-binaries/mc
    export PATH=$PATH:$HOME/minio-binaries/
    • For ARM64 (Apple silicon):
    curl https://dl.min.io/client/mc/release/linux-arm64/mc \
      --create-dirs \
      -o ~/minio-binaries/mc
    
    chmod +x $HOME/minio-binaries/mc
    export PATH=$PATH:$HOME/minio-binaries/

Configuring the MinIO client and migrating data

  1. Configure the endpoints you wish to use:

    mc config host add scw-fr-par https://s3.fr-par.scw.cloud <ACCESS_KEY> <SECRET_KEY> --api S3v4
    mc config host add scw-nl-ams https://s3.nl-ams.scw.cloud <ACCESS_KEY> <SECRET_KEY> --api S3v4
    Note

    Replace <ACCESS_KEY> and <SECRET_KEY> with the credentials of your API key.

  2. Optionally, add other providers:

    For Amazon S3-compatible storage:

    mc config host add s3 <ENDPOINT> <OTHER_ACCESS_KEY> <OTHER_SECRET_KEY> --api S3v4

    For Google Cloud Storage:

    mc config host add gcs  https://storage.googleapis.com <ACCESS_KEY> <SECRET_KEY> --api S3v2
  3. Create the buckets you need on the Scaleway Object Storage:

    mc mb --region fr-par scw-fr-par/<BUCKET_NAME>
    mc mb --region nl-ams scw-nl-ams/<OTHER_BUCKET_NAME>
  4. Start the migration of your data:

    mc cp --recursive s3/<BUCKET_NAME> scw-fr-par/<BUCKET_NAME>
    mc cp --recursive gcs/<OTHER_BUCKET_NAME> scw-nl-ams/<OTHER_BUCKET_NAME>

    The commands above:

    1. Migrates data from an Amazon S3-compatible Object Storage to Scaleway's fr-par Object Storage
    2. Migrates data from GCS Object Storage to Scaleway's nl-ams Object Storage
    Important

    Note that this migration can incur some cost from the Object Storage you are migrating from since they may bill egress bandwidth.

    The MinIO client displays a status bar during the transfer, allowing to observe the status of the migration:

For more information about MinIO Client refer to the official documentation.

Questions?

Visit our Help Center and find the answers to your most frequent questions.

Visit Help Center
No Results