Uploading and Deserializing protobuf data in AWS Lambda

Question

I have a requirement to send protobuf data to an AWS Lambda written in Node.js. I am experimenting with a "Hello World" example where I serialize and deserialize a Person messge. Example: person.proto

syntax = "proto3";

message Person {
    required int32 id = 1;
    required string name = 2;
    optional string email = 3;
}

Using Node.js and the package protobufjs I can generate the code from the proto file and serialize and deserialize a Person object to a file:

let person = personProto.Person.create();
person.id = 42;
person.name = 'Fred';
person.email = "fred@foo.com";
console.log("Person BEFORE Serialze=" + JSON.stringify(person1,null,2));

// Serialize
let buffer = personProtos.Person.encode(person1).finish();
console.log(buffer);
fs.writeFileSync("person.pb", buffer, "binary");

// Deserialize
let bufferFromFile = fs.readFileSync("person.pb");
let decodedPerson = personProtos.Person.decode(bufferFromFile);
console.log("Decoded Person=\n" + JSON.stringify(decodedPerson,null,2));

Output:

Person BEFORE Serialize={
  "id": 42,
  "name": "Fred",
  "email": "fred@foo.com"
}
<Buffer 08 2a 12 04 46 72 65 64 1a 0c 66 72 65 64 40 66 6f 6f 2e 63 6f 6d>
Decoded Person=
{
  "id": 42,
  "name": "Fred",
  "email": "fred@foo.com"
}

Using Postman, I want to upload the binary protobuf data to an AWS Lambda from the person.pb file and deserialize it in the Lambda. When I specify the body as "binary" type and specify the person.pb file, the person data shows up in the Lambda event body as:

"body": "\b*\u0012\u0004Fred\u001a\ffred@foo.com"

It looks like it got transformed into Unicode and encoded? How can I take the body string value and turn it back into the Node.js buffer:

<Buffer 08 2a 12 04 46 72 65 64 1a 0c 66 72 65 64 40 66 6f 6f 2e 63 6f 6d>

so that I can deserialize it back to the JSON object in my Lambda code?

I put the generated code from the .proto file into my Lambda so I can call:

let bufferFromEvent = event.body;  <== how do I get a buffer from this?
let decodedPerson = personProtos.Person.decode(bufferFromEvent);

Thanks

Hi Chris, Did you try var data = new Buffer.from(event.body, 'base64'); var message2 = personProtos.Person.decode(data); — Daniel Hernandez, Dec 18 '20 at 17:08

score 0 · Answer 1 · answered Apr 07 '22 at 21:18

The answer is what Daniel mentioned in the comment below your question. You need to use the Buffer class provided by Node.

Your lambda function will then look something like this.

const personProtos = require("./personProtos");

module.exports.handler = async event => {

const buffer = new Buffer.from(event.body);

console.log(personProtos.Person.decode(buffer).toObject());

return {
  statusCode: 200,
  body: "Success"
};

};

Uploading and Deserializing protobuf data in AWS Lambda

1 Answers1