Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. Buckets that are configured for object replication can be owned by the same AWS account or by different accounts. Object may be replicated to a single destination bucket or multiple destination buckets. Destination buckets can be in different AWS Regions or within the same Region as the source bucket.

S3 Cross-Region Replication (CRR) is used to copy objects across Amazon S3 buckets in different AWS Regions. CRR can help you do the following:

  • Meet compliance requirements — Although Amazon S3 stores your data across multiple geographically distant Availability Zones by default, compliance requirements might dictate that you store data at even greater distances. Cross-Region Replication allows you to replicate data between distant AWS Regions to satisfy these requirements.
  • Minimize latency — If your customers are in two geographic locations, you can minimize latency in accessing objects by maintaining object copies in AWS Regions that are geographically closer to your users.
  • Increase operational efficiency — If you have compute clusters in two different AWS Regions that analyze the same set of objects, you might choose to maintain object copies in those Regions.

In this post we going to learn how to replicate a s3 bucket across AWS Regions, for this lab I'm going to use US East Ohio and US West N.California, you can change it at your needs.

Requirements

  • Have an AWS account.

Hands-On

  1. First we going to create the buckets, on the AWS Management Console, go to S3

2. Click on create a bucket

3. Give it a name and select the Region, first I'm going to create Ohio Region

4. Enable Bucket Versioning

5. And click on create bucket

6. If everything is OK, you will see the Success message, click on Create bucket again to create the destination bucket

7. Give it a name, and select the destination Region

8. Enable Bucket Versioning

9. And click on Create bucket

10. Get the ARN of the buckets, the next step is going to create a IAM Role, and give it permissions in the two buckets, for that we need the Amazon Resource Name (ARN) select each bucket and click on Copy ARN, and write it on some text file

In my case

origin bucket is arn:aws:s3:::aws-versioning-test-origin

destination is arn:aws:s3:::aws-versioning-test-destination

11. Now go to Identity and Access Management (IAM)

12. Click on Roles

13. Click on Create role

14. On AWS service click on S3

15. And select S3 in "Select your case" and click on " Next: Permissions"

16. On the next window, click on Create policy

17. And click on the JSON tab

Replace the json with this, and put your own arn on the SourceBucket and DestinationBucket

{

   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:GetReplicationConfiguration",
            "s3:ListBucket"
         ],
         "Resource":[
            "arn:aws:s3:::SourceBucket"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[

            "s3:GetObjectVersion",
            "s3:GetObjectVersionAcl",
            "s3:GetObjectVersionTagging"

         ],
         "Resource":[
            "arn:aws:s3:::SourceBucket/*"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:ReplicateObject",
            "s3:ReplicateDelete",
            "s3:ReplicateTags"
         ],
         "Resource":"arn:aws:s3:::DestinationBucket/*"
      }
   ]
}

In my case the final json looks like this

{

   "Version":"2012-10-17",
   "Statement":[
      {
         "Effect":"Allow",
         "Action":[
            "s3:GetReplicationConfiguration",
            "s3:ListBucket"
         ],
         "Resource":[
            "arn:aws:s3:::aws-versioning-test-origin"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[

            "s3:GetObjectVersion",
            "s3:GetObjectVersionAcl",
            "s3:GetObjectVersionTagging"

         ],
         "Resource":[
            "arn:aws:s3:::aws-versioning-test-origin/*"
         ]
      },
      {
         "Effect":"Allow",
         "Action":[
            "s3:ReplicateObject",
            "s3:ReplicateDelete",
            "s3:ReplicateTags"
         ],
         "Resource":"arn:aws:s3:::aws-versioning-test-destination/*"
      }
   ]
}

18. Click on Review policy

19. Give the policy a name and description and click on Create policy

20. Create the policy and close the window, in the create role window, click on the refresh button, and search for the name you give it to the new policy, and click next

21. In the next screen click on Review

22. Give it a name to the role and click on Create role

23. Go to s3

24. Select the origin bucket

25. Click on Management tab

26. On Replication rules click on Create replication rule

27. Give it a name

26. In Source bucket select This rule applies to all objects in the bucket

27. In the destination bucket you can paste the arn of the destination or click on Browse S3

28. If you click on Browse S3, choose the bucket

29. In the IAM role select the role we create

30. Click on Save

31. Lets test the replication, go to S3, select the origin bucket and click on Upload

32. Select some file for testing, I'm selecting an image, and click Upload

33. You will see the Upload status

34. Go to buckets again, and click on destination, you will see the file there, it can be delay, but eventually you will see it there.

Great, now you know how to use the Cross-Region Replication.

In the next post we going deeper into the AWS Services.

Clean-UP

  1. Select each bucket and click on Empty

2. Confirm

3. Select each bucket again and this time click on Delete

4. Confirm

5. Go to IAM

6. Select Roles

7. Select the role we create and click on Delete role

8. Confirm

9. Go to policies

10. Search for the policy we create, and click on Policy actions, Delete

11. Confirm