4

I have a firebase function which I use as a webhook for sendgrid's inbound parse webhook. So that means whenever an email is sent to my domain, it calls the webhook. I know the webhook is being called, but I can't get to the data being sent by Sendgrid. This link states that all the information (text, sender, ect) should be right there in the headers. However when I print out req.headers I get this:

{ host: 'us-central1-project-name.cloudfunctions.net',
  'user-agent': 'Sendlib/1.0 server.sendgrid.net',
  'transfer-encoding': 'chunked',
  'content-type': 'multipart/form-data; boundary=xYzZY',
  forwarded: 'for="ip";proto=https',
  'function-execution-id': 'id',
  'x-appengine-city': '?',
  'x-appengine-citylatlong': '0.000000,0.000000',
  'x-appengine-country': 'US',
  'x-appengine-default-version-hostname': ~~~~~~~~~~~~~.appspot.com',
  'x-appengine-https': 'on',
  'x-appengine-region': '?',
  'x-appengine-request-log-id': 'super-long-id',
  'x-appengine-user-ip': 'ip',
  'x-cloud-trace-context': 'id/number;o=1',
  'x-forwarded-for': 'ip',
  'x-forwarded-proto': 'https',
  'accept-encoding': 'gzip',
  connection: 'close' }'

(Obviously I replaced all the ID's and everything)

Where is the email information? I have tried doing all of the following and none of them produced any information regarding the email. 

exports.reciever = functions.https.onRequest((req, res) => {
  try {
   console.log("Email recieved");
   console.log(req);
   console.log(req.headers);
   console.log(req.header.to);
   console.log(req.body);
   console.log(req.get('to'));
   console.log(req.body.to);
   console.log(req.rawBody);
   console.log(req.query);
   console.log(req.query.to);
   console.log(req.params);
   console.log(req.path);
   console.log(req.rawBody);
  } catch (e) {}
  finally {
      res.send("2xx");
  } 
})
Mr. Arbitrary
  • 932
  • 11
  • 25
  • The documentation says that the email information is in the body of the request, not the headers. I'd expect the data you're looking for to be in req.body or req.rawBody. – Doug Stevenson Jan 02 '20 at 00:21
  • the body object produces this wierd string followed by a series of characters busted into groups of 2. Do I need to decode this? – Mr. Arbitrary Jan 02 '20 at 00:29
  • 1
    Yes, I would assume this node buffer contains the data in the post body as described by the documentation. It's probably multipart data that needs special decoding. – Doug Stevenson Jan 02 '20 at 00:45
  • Formidable produces an error when parsing it though. It says cannot read property 'content-length' of undefined – Mr. Arbitrary Jan 02 '20 at 15:57

3 Answers3

3

So, it turns out it's actually very very simple. Add

.toString()

At the end of the req.body or req.rawBody object.

Mr. Arbitrary
  • 932
  • 11
  • 25
  • Did you manage to get this working? When I do req.body.toString() I just get a long string it seems. How did you extract for example the subject and text? @A.S.H – Gnopps Mar 10 '20 at 13:49
  • 1
    This is where formidable or something else comes in. You still need to parse the stringified buffer after the `.toString()`. It is a tedious process to do alone, so I highly recommend a node library for parsing MIMEs – Mr. Arbitrary Mar 10 '20 at 14:30
  • What lib did you use for this? – Grant Singleton Aug 04 '21 at 18:59
1

The issue is about to enable use in serverless environments (AWS Lambda, Firebase/Google Cloud Functions, etc.) and environments where the request has already been processed. Parse an email with MIME is not a good idea. Thus, here my solution:

import * as functions from 'firebase-functions';
import * as express from 'express';
const formidable = require('formidable-serverless');

export class InboundEmail {
  constructor(from, subject, text, html, to) {}
  doStrategy() {
     //Store inbound email
     //Send outbound email
     //...
  }
}

export class EmailPostWebHook {
  private form = new formidable.IncomingForm();
  private incomeEmail: IncomeEmail;

  async run(request: express.Request, res: express.Response) {
    try {
      this.parse(request);
      await this.incomeEmail.doStrategy();
    } catch (e) {
      console.log(e);
    }
    return res.sendStatus(200);
  }

  private parse(request: express.Request) {
    this.form.parse(request, (errors: any, fields: any) => {
      this.incomeEmail = new IncomeEmail(
        fields.from
        fields.subject,
        fiels.text
        fields.html,
        fields.to
      );
    });
  }
}


const app = express();

const emailPostWebHook = new EmailPostWebHook();
app.post('/', emailPostWebHook.run.bind(emailPostWebHook));

export const InboundEmailHook = functions
  .runWith({
    timeoutSeconds: 30,
    memory: '2GB',
  })
  .https.onRequest(app);

package.json

{
  "name": "temp",
  "main": "lib/index.js",
  "dependencies": {
    "express": "^4.17.1",
    "firebase": "^7.15.1",
    "firebase-admin": "^8.12.1",
    "firebase-functions": "^3.7.0",
    "formidable-serverless": "^1.0.3"
  },
  "devDependencies": {
    "@firebase/firestore-types": "^1.11.0",
    "@types/node": "^14.0.13",
    "@types/express": "^4.17.6",
    "tslint": "^6.1.2",
    "typescript": "^3.9.5"
  },
  "engines": {
    "node": "10"
  },
  "private": true
}

References:

https://github.com/sendgrid/sendgrid-nodejs/blob/master/docs/examples/webhooks-docker/routes/inbound-parse.js

https://github.com/Amit-A/formidable-serverless/

https://www.npmjs.com/package/formidable

Carlos Eduardo Sanchez Torres
  • 90
  • 6
0

I finally managed to get it working by installing Busboy in the functions-directory and then using the code provided here: https://stackoverflow.com/a/54026200/10372124

Gnopps
  • 339
  • 4
  • 14