HomeStorageObject StorageAPI/CLI
Migrating data from one bucket to another
Update content

Migrating data from one bucket to another

Reviewed on 13 June 2023 • Published on 14 September 2018
Security & Identity (IAM):

You may need certain IAM permissions to carry out some actions described on this page. This means:

  • you are the Owner of the Scaleway Organization in which the actions will be carried out, or
  • you are an IAM user of the Organization, with a policy granting you the necessary permission sets
Requirements:
  1. Create a new bucket.

    aws s3api create-bucket --bucket BUCKET-TARGET
  2. Copy the objects between the S3 buckets.

    Important:

    If you have objects in the Scaleway Glacier storage class you must restore them before continuing.

    aws s3 sync s3://BUCKET-SOURCE s3://BUCKET-TARGET
  3. Check that the content of the copied objects matches the content of the source ones.

    aws s3 ls --recursive s3://BUCKET-SOURCE --summarize > bucket-contents-source.txt
    aws s3 ls --recursive s3://BUCKET-TARGET --summarize > bucket-contents-target.txt
  4. Compare objects that are in the source and target buckets by using the outputs that are saved to files in the AWS CLI directory. See the following example output:

    aws s3 ls --recursive s3://DOC-EXAMPLE-BUCKET --summarize

    Sample output:

    2022-01-20 11:20:49 multiaz.png
    Total Objects: 1 Total Size: 18640
  5. Update any existing applications or workloads so that they can use the target bucket name, and delete the source bucket.

    Important:

    Delete the old bucket once everything is copied to avoid additional charges. You can use the following command: aws s3 rb s3://$bucket-source --force

See Also