Correct code to upload local file to S3 proxy of API Gateway

Question

I created an API function to work with S3. I imported the template swagger. After deployment, I tested with a Node.js project by the npm module aws-api-gateway-client.

It works well with: get bucket lists, get bucket info, get one item, put a bucket, put a plain text object, however I am blocked with put a binary file.

firstly, I ensure ACL is allowed with all permissions on S3. secondly, binary support also added image/gif application/octet-stream

The code snippet is as below. The behaviors are:

1) after invokeAPI, the callback function is never hit, after sometime, the Node.js project did not respond. no any error message. The file size (such as an image) is very small. 2) with only two times, the uploading seemed to work, but the result file size is bigger (around 2M bigger) than the original file, so the file is corrupt.

Could you help me out? Thank you!

var filepathname = './items/';
var filename = 'image1.png';

fs.stat(filepathname+filename, function (err, stats) {

    var fileSize = stats.size ;

    fs.readFile(filepathname+filename,'binary',function(err,data){ 

        var len = data.length;
        console.log('file len' + len);

        var pathTemplate = '/my-test-bucket/' +filename ; 
        var method = 'PUT';

        var params = {
            folder: '',
            item:''
        };
        var additionalParams = {
            headers: {
             'Content-Type': 'application/octet-stream',
              //'Content-Type': 'image/gif',
             'Content-Length': len
           }
        };

        var result1 = apigClient.invokeApi(params,pathTemplate,method,additionalParams,data) 
        .then(function(result){
            //never hit :(
            console.log(result);
         }).catch( function(result){
              //never hit :(
             console.log(result);
        });;

    }); 

});

Answer 1

We encountered the same problem. API Gateway is meant for limited data (10MB as of now), limits shown here,

http://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

Self Signed URL to S3:

Create an S3 self signed URL for POST from the lambda or the endpoint where you are trying to post.

How do I put object to amazon s3 using presigned url?

Now POST the image directly to S3.

Presigned POST:

Apart from posting the image if you want to post additional properties, you can post it in multi-form format as well.

http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#createPresignedPost-property

If you want to process the file after delivering to S3, you can create a trigger from S3 upon creation and process with your Lambda or anypoint that need to process.

Hope it helps.