Uploading photos to AWS S3 — getting started with Cognito and IAM

While Amazon’s Simple Storage Service (S3) is powerful and easy to use once it’s up and running, it can be a pain to set up if you’ve never used a service like it before. When I implemented S3 in an application for the first time I found myself tied in a knot of documentation and dashboards, trying to figure out what I needed just to allow users of my application to upload images to my bucket. I thought I might try to save someone else the time that I spent figuring it all out, so here is my step-by-step guide to getting up and running with Amazon’s cloud storage service.

A bit of context

This implementation of S3 allows a user to upload photos to an S3 bucket, directly from the browser. By using the AWS SDK and Amazon’s identity services (more below), it’s possible to avoid processing the images on your own back-end, saving a potential boatload of your own resources. A user can log into my application, select files via a form, and upon submission of the form the files are validated, and then sent off to my S3 bucket. Sounds simple enough, right?

The services you’ll need

In order to get this all working smoothly, you’ll need to use 3 different amazon services*:
S3 — This is where all of your images (or objects as amazon collectively calls what you upload) will be stored.
Cognito — You use this to create Identity Pools, which are used to provide temporary, limited privilege credentials so that your users can access your AWS services.
Identity and Access Management (IAM)— This is for creating policies that govern the roles that you create in Cognito, controlling what your users can and cannot use/access.

*This is NOT the only way to use S3, but as far as I can work out it’s the easiest.

How they all slot together

This was arguably the thing that was hardest for me to get my head around. What order do I need to set everything up so that all of the services have everything they need from each other? There was so much documentation across the three services that I thought my head would explode. When I finally pieced it all together this is what it all boils down to:

We’ll get to Federated Identities, permission policies and ARN’s in a moment, but for now, this is the flow you need to use to make sure everyone has everything they need, so that you aren’t going to have any Access denied errors, or anyone being able to do more with your AWS account than you intended.

TL;DR just tell me how to do it and give me the code I need

1. Create an Amazon Web Services account — this gives you access to all of the services I’ve spoken about above. You need to hook up a bank account but the free tier should be plenty for anyone not using the service for commercial purposes.
2. Make sure that your region is set to wherever you want to host your bucket, if you don’t you’ll have to re-do basically all of the succeeding steps with the correct region. This could be done in the top right of the screen at the time of writing this post.
3. Go to S3 and create a new bucket. Don’t change any of the settings just yet, we’ll configure them when we’ve set up Cognito and IAM. Just click through until you can see your bucket in the dashboard.
4. Go to Cognito. Click Manage Federated Identities, and create a new Identity Pool. Make sure you allow access to unauthorized identities. Your users will all have the same level of access and if you have implemented some kind of authentication (for example using O-Auth) then there is no need to re-implement it through Amazon. However, if you would like to add authorization through amazon as well, you can apply this at the stage you’re at now.
5. Allow Cognito to access IAM — you can’t finish creating the identity pool without this. You’re now done with Cognito. It’s that simple. The roles that were generated are automatically accessible by IAM.
6. Go to IAM. We need to add a set of permissions to the roles that we just created in Cognito. You are able to select a set of permissions that has been pre-defined by Amazon, such as S3FullAccess, which allows a user to perform any action on any resource within S3. The policies can be accessed from the nav on the left of the page. If this is ok for your use-case, attach it to the role by clicking ‘Attached Entities’, ‘Attach’ and select the ‘Unauth_Role’. If not, and you want to create your own, as I did, see points 7 and 8.
7. As I already know that for my application users need only be able to upload photos and delete the photos they have uploaded, I created a minimum-privilege policy to allow them perform those actions and those actions alone. To do this, go to Create Policy, and select the permissions that you want to give to your users.
8. The permissions I selected were PutObject, PutObjectAcl, and DeleteObject. I also restricted my users to only allowing actions on Specific resources, and selected ‘Any’ to allow permissions on any Object. Leave the conditions blank, as we can configure the bucket with conditions later. With these options you can create your policy and attach it to your role, but if you prefer it in JSON form, I’ve attached what mine looked like below point 9!
9. Finally, go to Users and create a user, to which you can also attach the policy. This may seem like a redundant step as we’ve configured our role and policy, but we need it to attach to the bucket policy in the next section.

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "S3MinPriv", <= You can write what you like here
         "Effect": "Allow",
         "Action": [
            "s3:PutObject",
            "s3:PutObjectAcl",
            "s3:DeleteObject"
         ],
         "Resource": "arn:aws:s3:::BUCKET_NAME/*"
      }
   ]
}

Adding it all to the bucket — via the bucket policy

If you click on your bucket, you are able to configure your Bucket Policy and Cross Origin Resource Sharing (CORS) settings. The bucket policy is used as a blanket policy for anyone trying to access your objects, regardless of the role policy governing them. You can use the Policy Generator to create it, or copy the one I used.

Sample bucket policy

User ARN: IAM > Users > Select User, the user ARN can be found at the top of the page.
Resource ARN: S3 > Select Bucket > a blue modal should appear, with a button in the top right that allows you to copy the Bucket ARN.

The CORS configuration protects you from malicious users trying to access your resources from domains that are not your own. If it’s not configured properly then you could be either vulnerable to someone using your resources and racking up a huge AWS bill, or making it so that your own users can’t access them. I have attached my config below, make sure you change your domain (in the AllowedOrigin property).

Configuring AWS in the browser

This is the last step before you can be on your way. I am using React, so in order to use the SDK you will need to npm install --save aws-sdk and import the module in your component/service. If you aren’t using React you can include the following script tag in the head of your document:

<script src="https://sdk.amazonaws.com/js/aws-sdk-2.224.1.min.js"></script>

When everything is set up, then the browser configuration is relatively straight forward. The last thing that was a little tricky to find is the Identity Pool Id for the AWS credentials. If you go to the sample code in Cognito, you can find an example (if you go into the dashboard for the Identity Pool that you created before). Here’s the full setup:

Example browser config

If you are anything like me when I was setting this up this will be the sixth or seventh example you’ve looked at, but if you can take just one or two snippets from this to save yourself some time then I’ll consider my job done!

Happy clouding and leave me a comment if you have any questions that I might be able to help you with!

To anyone getting started with Lambda I have just written a piece on using it to automate the compression of user-submitted images, check it out if you think it could help you!