AWS API Gateway base64Decode produces garbled binary?

Question

I'm trying to return a 1px gif from an AWS API Gateway method.

Since binary data is now supported, I return an image/gif using the following 'Integration Response' mapping:

$util.base64Decode("R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7")

However, when I look at this in Chrome, I see the following binary being returned:

Instead of:

Could anyone help me understand why this is garbled and the wrong length? Or what I could do to return the correct binary? Is there some other what I could always return this 1px gif without using the base64Decode function?

Many thanks in advance, this has being causing me a lot of pain!

EDIT

This one gets stranger. It looks like the issue is not with base64Decode, but with the general handling of binary. I added a Lambda backend (previously I was using Firehose) following this blog post and this Stack Overflow question. I set images as binaryMediaType as per this documentation page.

This has let me pass the following image/bmp pixel from Lambda through the Gateway API, and it works correctly:

exports.handler = function(event, context) {

  var imageHex = "\x42\x4d\x3c\x00\x00\x00\x00\x00\x00\x00\x36\x00\x00\x00\x28\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x01\x00\x18\x00\x00\x00\x00\x00\x06\x00\x00\x00\x27\x00\x00\x00\x27\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\x00\x00";
  context.done(null, { "body":imageHex });

};

However the following images representing an image/png or a image/gif get garbled when passed through:

exports.handler = function(event, context) {

//var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00\xff\xff\xff\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x01\x44\x00\x3b";
//var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\xff\xff\xff\x00\x00\x00\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3b";
  var imageHex = "\x47\x49\x46\x38\x39\x61\x01\x00\x01\x00\x80\x00\x00\x00\x00\x00\x00\x00\x00\x21\xf9\x04\x01\x00\x00\x00\x00\x2c\x00\x00\x00\x00\x01\x00\x01\x00\x00\x02\x02\x44\x01\x00\x3b\x0a"
  context.done(null, { "body":imageHex });

};

This seems to be the same issue as another Stack Overflow question, but I was hoping this would be fixed with the Gateway API binary support. Unfortunately image/bmp doesn't work for my use case as it can't be transparent...

In case it helps anyone, this has been a good tool for converting between base64 and hex.

Thanks, the response headers are: `HTTP/1.1 200 OK Content-Type: image/gif Content-Length: 52 Connection: keep-alive Date: Mon, 02 Jan 2017 22:08:55 GMT x-amzn-RequestId: 0d3f620c-d138-11e6-941a-0f16afc9bdc4 X-Amzn-Trace-Id: Root=1-586acf77-93ce6c87faa62ee76758abf5 X-Cache: Miss from cloudfront Via: 1.1 227087338674ca3d3d23a79539f2998b.cloudfront.net (CloudFront) X-Amz-Cf-Id: 9V1XUr1cPqjm7Bj2HOFfakLlFM5MWo_Ucuv9cdk35xsBz_xhcPvixQ==` And response comes back as: `data:image/gif;base64,R0lGODlhAQABAO+/vQAAAAAA77+977+977+9Ie+/vQQBAAAAACwAAAAAAQABAAACAUQAOw==` — rjmurt, Jan 02 '17 at 22:09

score 34 · Answer 1 · answered Dec 12 '17 at 20:22

To anyone else having problems with this: I was also banging my head against the wall trying to retrieve a binary image over API Gateway proxy integration from lambda, but then I noticed that it says right there in the Binary Support section of Lambda Console:

API Gateway will look at the Content-Type and Accept HTTP headers to decide how to handle the body.

So I added Accept: image/png to the request headers and it worked. Oh the joy, and joyness! No need to manually change content handling to CONVERT_TO_BINARY or muck about with the cli. Of course this rules out using, for example, <img src= directly (can't set headers).

So, in order to get a binary file over API Gateway from lambda with proxy integration:

List all supported binary content types in the lambda console (and deploy)
The request Accept header must include the Content-Type header returned from the lambda expression
The returned body must be base64 encoded
The result object must also have the isBase64Encoded property set to true

Code:

callback(null, {
    statusCode: 200,
    headers: { 'Content-Type': 'image/png' },
    body: buffer.toString('base64'),
    isBase64Encoded: true
}

This worked for me. In addition, you could set the Binary Media Types (in the Settings of the API) to `*/*` and then you won't need to set an Accept header. However the caveat is that this prevents you from using text output. — istvanp, Apr 13 '18 at 04:04
This response object returns a 500 Internal Server error response — Benjamin Heinke, Jan 13 '21 at 08:35

score 20 · Accepted Answer · edited Nov 04 '22 at 13:42

20

It looks like this was a known issue previously: https://forums.aws.amazon.com/thread.jspa?messageID=668306&#668306

But it should be possible now that they've added support for binary data: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-payload-encodings.html

It looks like this is the bit we need: "Set the contentHandling property of the IntegrationResponse resource to CONVERT_TO_BINARY to have the response payload converted from a Base64-encoded string to its binary blob". Then we shouldn't need the base64Decode() function.

Working on a test now to see if this works.

EDIT: I was finally able to get this working. You can see the binary image here: https://chtskiuz10.execute-api.us-east-1.amazonaws.com/prod/rest/image

I updated the method response as follows:

I updated the integration response to include a hard-coded image/png header:

The last step was tricky: setting the contentHandling property to "CONVERT_TO_BINARY". I couldn't figure out how to do in the AWS console. I had to use the CLI API to accomplish this:

aws apigateway update-integration-response \
    --profile davemaple \
    --rest-api-id chtskiuzxx \
    --resource-id ki1lxx \
    --http-method GET \
    --status-code 200 \
    --patch-operations '[{"op" : "replace", "path" : "/contentHandling", "value" : "CONVERT_TO_BINARY"}]'

I hope this helps.

edited Nov 04 '22 at 13:42

Greg

2,559
4
28
46

answered Jan 02 '17 at 23:04

Dave Maple

8,102
4
45
64

1

Thanks for your help Dave, I've added further details of my investigation above. Would be really interested to hear how you get on! – rjmurt Jan 03 '17 at 08:48
let me know if you get this working once you have a chance to review. – Dave Maple Jan 03 '17 at 14:07
1

Wow, that worked! I can't thank you enough. I would not have spotted that! – rjmurt Jan 03 '17 at 19:46
seriously -- the docs could use a little update in this area (is fairly new functionality in their defense). – Dave Maple Jan 03 '17 at 19:48
@tkiethanom : it does work if the response body is a valid base64 encoded image, the method response headers are set, content-type is set to match the image type and you've performed the patch operation to set contentHandling to CONVERT_TO_BINARY. I did get that error at one point but I think it resolved after the patch update. – Dave Maple Jan 06 '17 at 02:01
2

Actually this worked for me. So the CLI command is basically to get around the fact that when you select Lambda Function from the Create Method screen you aren't able to set the Content Handling dropdown. The dropdown only appears when you select AWS Service. – tkiethanom Jan 06 '17 at 02:03
@tkiethanom :: that's exactly it. It is available in an aws service integration -- but doesn't appear as an option in the console in a lambda integration. – Dave Maple Jan 06 '17 at 02:06
After further exploration I haven't been able to get this to work when checking the Lambda Proxy Integration box. Luckily I can use my Lambda without Proxy Integration, but I wanted to point this out in case anyone else runs into this issue. – tkiethanom Jan 06 '17 at 23:49
@tkiethanom : will try to setup a test this weekend to see what's different in the proxy integration. – Dave Maple Jan 06 '17 at 23:52
@DaveMaple Have you had any luck setting the Content-Type dynamically? When the response is returned as binary the API gateway is having trouble extracting out my integration.response.body.contentType for obvious reasons. – tkiethanom Jan 17 '17 at 18:26
i've tried a lot of things @tkiethanom :: so far sol. without an integration response i haven't found a way. – Dave Maple Jan 17 '17 at 22:47
ok sorry @tkiethanom: I am able to set a content-type to `image/png` using the lambda proxy, see gist here: https://gist.github.com/davemaple/47cf6e4dc8b1e23c3562c41a900a69f0. I just can't get the response converted to binary. – Dave Maple Jan 17 '17 at 23:23
@DaveMaple I'm in a similar situation, when I use Lambda Proxy to set Content-Type the CONVERT_TO_BINARY setting doesn't work anymore. So I'm trying to figure out a way to have dynamic Content-Type exported as binary. – tkiethanom Jan 18 '17 at 00:23
1

I want to add contentHandling property to "CONVERT_TO_BINARY" in integration response, but could not find any way in aws web console. Any one did that or any other workaround, please tell. – Hassan Siddique Jan 23 '17 at 15:00
You can do it in with the AWS skd or cli @HassanSiddique -- but not in the web interface. The accepted answer has a n example using the cli. – Dave Maple Jan 23 '17 at 15:20
Could you please explain the last step? I didn't know how to find aws cli and how to add the contentHandling property to "CONVERT_TO_BINARY". – Sabreena May 02 '17 at 10:57
@Sabreena -- try these steps: http://docs.aws.amazon.com/cli/latest/userguide/installing.html – Dave Maple May 02 '17 at 13:13
@Dave Maple -- Can i send audio file as multipart/form-data through api gateway to s3? please check http://stackoverflow.com/questions/43733254/how-to-upload-audio-file-to-s3-via-api-gateway – Sabreena May 03 '17 at 06:29

score 1 · Answer 3 · answered Nov 21 '17 at 17:39

1

Check out this answer. It helped me with exposing PDF file for download through GET request without any additional headers.

answered Nov 21 '17 at 17:39

megapixel23

829
1
10
22

AWS API Gateway base64Decode produces garbled binary?

3 Answers3

Linked

Related