Importing a copy of S3 Objects from one bucket to another

Object Storage allows you to import a copy of your objects in a bucket to another bucket using the AWS-CLI.

The sync command is used to synchronize directories and S3 prefixes.

This command will:

• Recurringly copy new and updated files from the source directory to the target one
• Use the CopyObject APIs to list objects in the source and target buckets and identify which ones they have in common
• Compare both to determine disparities in LastModified dates between the same objects

Importing copied objects from the same project

To import copied objects to a bucket in the same Project, use the aws s3 sync command.

aws s3 sync s3://$SourceBucket s3://$TargetBucket
Copy code

Replace $SourceBucket with the name of the bucket where the copy is currently stored, and $TargetBucket with the name of the bucket the copy will be stored in.

Importing copied objects from a different project

To import a copy of objects in a bucket to another bucket in a different Project, you need to implement a bucket policy.

A bucket policy is a resource-based policy option. In this context, bucket policies are used to grant the target Project access to the source bucket.

1. Create a file with your bucket policy for the source bucket.

   Note:

   Bucket policies use a JSON-based access policy language. You can find more details about the JSON policy grammar on this page.

   {
           "Version": "2012-10-17",
           "Id":"MyBucketPolicy",
           "Statement": [
                   {
                           "Sid": "DelegateS3Access",
                           "Effect":"Allow",
                           "Principal":{
                                   "SCW":"project_id:<TARGET_PROJECT_ID>"
                           },
                           "Action":[
                                   "s3:ListBucket",
                                   "s3:GetObject"
                           ],
                           "Resource":[
                                   "<BUCKET_NAME>",
                                   "<BUCKET_NAME>/*"
                           ]
                   }
           ]
   }
   
   Copy code

2. Run the following command:

   $ aws s3api put-bucket-policy --bucket <SOURCE_BUCKET> --profile <SOURCE_PROJECT> --policy file://bucket-policy.json
   Copy code

3. If necessary, create the target bucket in the target project.

   $ aws s3api create-bucket --bucket <TARGET_BUCKET> --profile <TARGET_PROJECT>
   {
     "Location": "/<TARGET_BUCKET>"
   }
   Copy code

4. Copy objects and import using the sync command.

5. Replace <SOURCE_BUCKET> with the name of the bucket located in the source project and <TARGET_BUCKET> with the name of the bucket in the target project.

   $ aws s3 sync s3://<SOURCE_BUCKET> s3://<TARGET_BUCKET> --profile <TARGET_PROJECT>
   Copy code