Saving base64 string to Amazon S3

Question

I'm working on a React Native application where I'm trying to take images from a user's camera roll, convert them to a base64 string and store them to Amazon S3 for later use.

Following this blog post I'm able to take a user's camera roll and convert the images to base64: react-native-creating-a-custom-module-to-upload-camera-roll-images

I'm then sending the base64 string image data to a simple Express server I have set up to post the data to my Amazon S3 bucket.

// Only getting first img in camera roll for testing purposes
CameraRoll.getPhotos({first: 1}).then((data) => {

  for (let i = 0; i < data.edges.length; i++) {
    NativeModules.ReadImageData.readImage(data.edges[i].node.image.uri, (imageBase64) => {

      // Does the string have to be encoded?
      // const encodeBase64data = encodeURIComponent(imageBase64);

      const obj = {
        method: 'POST',
        headers: {
          'Accept': 'application/json',
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          'img': imageBase64
        })
      }

      fetch('http://localhost:3000/saveImg', obj)
        .then((res) => {
          console.log(JSON.parse(res._bodyInit));
        })

    })
  }

My imageBase64 variable in this instance is a pretty large string reading like: /9j/4AAQSkZJRgABAQAASABIAAD/4QBYRXhpZgAATU0AKgAAA...abX+Yub/API3zf8A7G2Z/wDqdiD/AExyf/kT5R/2Kst/9QqB0x6H6GuBbr1R6D2foz+ZT/gof/yep8bf934f/wDqC6PX96+Cn/JruFf+6z/6t8UfwP4wf8nM4n9Mq/8AVbRPjOv1I/OAoA//2Q== With the ... being several more characters.

I'm sending this base64 string to my express server and posting the data:

app.post('/saveImg', function(req, res) {

  // this will be moved once testing is complete
  var s3Bucket = new AWS.S3( { params: {Bucket: '[my_bucket_name]'} } );

  // Do I need to append this string to the image?
  var baseImg = 'data:image/png;base64,' + req.body.img;

  var data = {
    Key: test_img,
    Body: req.body.img,
    ContentEncoding: 'base64',
    ContentType: 'image/png'
  };

  s3Bucket.putObject(data, function(err, data){
   if (err) {
     console.log(err);
     console.log('Error uploading data: ', data);
   } else {
     res.send(data);
     console.log('successfully uploaded the image!');
   }
 });
  // res.send(base64data)
});

I successfully send the data to Amazon S3 and see my image file in the bucket however when I try to visit the link to see the actual image itself, or pull it into my React Native app, I get nothing.

ie If I visit the url to test_img above after it's in Amazon S3 I get:

https://s3.amazonaws.com/my_bucket_name/test_img
This XML file does not appear to have any style information associated with    it. The document tree is shown below.
<Error>
  <Code>AccessDenied</Code>
  <Message>Access Denied</Message>
  <RequestId>BCE6E07705CF61B0</RequestId>
    <HostId>
aF2l+ucPPHRog1QaaXjEahZePF0A9ixKR0OTzlogWFHYXHUMUeOf2uP7D/wtn7hu3bLWG8ulKO0=

     </HostId>
</Error>

I've uploaded images manually to this same bucket and their links appear fine, and I'm additionally able to pull them into my React Native application with no problem for viewing.

My question is what am I doing wrong between getting the base64 string data and sending it to my Express server for saving to my bucket? Does the base64 string have to be encoded? Do I need to convert the base64 string to a Blob before sending it to Express?

Thanks for the help!

Answer 1

I just ran into the same issue. You have to convert the base64 string to a Blob before uploading to S3.

This answer explains how to do this conversion. Using node-fetch, Here's how to integrate in your example :

require('node-fetch')

app.post('/saveImg', function(req, res) {

  // this will be moved once testing is complete
  var s3Bucket = new AWS.S3( { params: {Bucket: '[my_bucket_name]'} } );

  var imageUri = 'data:image/png;base64,' + req.body.img;

  fetch(imageUri)
    .then(function(res){ return res.blob() })
    .then(function(image){
      var data = {
        Key: test_img,
        Body: image,
        ContentEncoding: 'base64',
        ContentType: 'image/png'
      };

      s3Bucket.putObject(data, function(err, data){
       if (err) {
         console.log(err);
         console.log('Error uploading data: ', data);
       } else {
         res.send(data);
         console.log('successfully uploaded the image!');
       }
     });
   })
});

Once that's done, you may then preview the uploaded image on S3 or pull it into your app.

Answer 2

It's a permission thing and has nothing to do with ReactNative nor Base64-enconding.

You've got an "AccessDenied"-Error, that means that the image isn't publicly available. Only if you configure your bucket with the right permissions (or even the specific file, i'll explain below), you will receive the content of an image without having signed-urls.

To investigate if this is the root cause you can try to make an image public in the s3-console. Just go to your s3-bucket and have a right-mouse-click on an image-file:

In the context-menu are two interesting items listed for you: "make public", and "open".

If you choose "open", you'll get a "signed url" to the file, which means that the plain url to the image will be appened with specific parameters to make this file public available for a while:

Also you can try out "make public" and reload your image-url again to see if it will be available now for you.

1. Approach, bucket-wide:

One solution is to create an IAM-Policy for the whole bucket to make every object in it public:

{
  "Version": "2008-10-17",
  "Statement": [{
    "Sid": "AllowPublicRead",
    "Effect": "Allow",
    "Principal": {
      "AWS": "*"
    },
    "Action": [ "s3:GetObject" ],
    "Resource": [ "arn:aws:s3:::YOUR_BUCKET_NAME/*" ]
  }]
}

So go to your bucket in AWS console, click on the bucket, and on the right pane open up "permissions". You can create a new policy like the one above.

2. Second solution, object-specific

Another approach would be to add ACL-specific headers to the putObject-Method:

'ACL'          => 'public-read'

I don't know your backend-sdk, but i'll guess something like this:

var data = {
    Key: test_img,
    Body: req.body.img,
    ContentEncoding: 'base64',
    ContentType: 'image/png',
    ACL: 'public-read',
  };

I just added the ACL-specific line here. Depending of the SDK it could be necessary to use the plain aws-headers "x-amz-acl: public-read" instead of "ACL:public-read". Just try both.

Answer 3

Adding Bucket policy to your bucket will resolve the issue.