1

I'm calling this Lambda function via API Gateway. My issue is that the image file is malformed, meaning that it does not open.

import boto3
import json

def lambda_handler(event, context):
    print(event)

    # removing all the data around the packet
    # this also results in a malformed png
    start = '<?xpacket end="r"?>'
    end = '\r\n------'
    content = str(event['body'])
    content = content[content.index(start) + len(start):content.index(end)].encode('utf-8')
    
    bucket_name = "bucket-name"
    file_name = "hello1.png"
    lambda_path = "/tmp/" + file_name
    s3_path = file_name

    s3 = boto3.resource("s3")
    s3.Bucket(bucket_name).put_object(Key=s3_path, Body=content)
    
    return {
        'statusCode': 200,
        'headers': {
            'Access-Control-Allow-Origin': '*',
        },
        'body': json.dumps(event)
    }
Manuel Vasquez
  • 13
  • 1
  • 5
  • I've had similar issues with malformed images, in my case it was because API Gateway uses base64 encoding for multipart/form-data by default. Because I needed to process the images while they were being uploaded pre-signed URLs weren't an option for my use case. This answer helped point me in the right direction: https://stackoverflow.com/a/41770688/12992963 Could you share the log output (from uploading a small image) that your lambda generates? – Florian Franken Feb 17 '21 at 08:52

1 Answers1

4

Lambda has payload limit of 6mb for synchronous call and 256KB for async call.

https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html

Also api gateway has limit of 10MB for RESTfull APIs and 128KB for socket message

https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html

It may be the primary reason why some part of file is uploaded, some not.

Even if you do not hit those limits with smaller file sizes, you pay for lambda execution while uploading. It is just a waste of lambda's time.

Also there may be config on api gateway to modify payload while pushing it to lambda. Make sure there is no active template which would convert the request before hitting lambda and check if use lambda as proxy is checked at gateway-api dashboard for this resource.

To upload to S3 better use Pre-Signed URL for an Amazon S3 PUT Operation:

https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-presigned-urls.html

Lukas Liesis
  • 24,652
  • 10
  • 111
  • 109
  • 1
    I looked into pre-signed URLs and it seems they might solve my issue. I'll check it out and post my findings here. Thanks! – Manuel Vasquez Feb 15 '21 at 13:46