1

My API needs to handle file uploads as well as attached files in emails. Emails are routed to my API via the mail parse function at Sendgrid.

Both methods results in a multipart form. In the email case, fields are "to", "from", etc. In the ordinary upload form I decide the fields.

I don't want to handle local temp files, so ideally I'll just pipe the file streams to S3. I have implemented this and it works great. But there are edge cases..

Before I pipe the files to S3 I want to ensure that the email is legit. I do this by checking both the from and to addresses. Unfortunately multipart forms does not specify a specific order for fields/files, rather the spec says to preserve order of original form.

Problem

Occationally I have received emails from Sendgrid parse where the fields arrive AFTER the files.

So, in order to validate the sender via the form fields I might need to buffer the file while waiting for the field data. This seems to be a quite stupid way of handling incoming files, i.e. start to process/store files before I even know if I should handle them.

All emails sent to a specified subdomain will be forwarded from Sendgrid to my server. This means that I could potentially be flooded with emails which I'd have to handle completely (including buffering the files) before I can decide if the email is legit.

Question

Have I misunderstood something?

Some email clients postpones downloading of attachments. How is this done?

The tools I'm using

  • node.js

  • multiparty

  • Sendgrid Parse

Michael
  • 1,764
  • 2
  • 20
  • 36

1 Answers1

1

At last I grasped this. The simple answer is that a multipart form is a single stream. If files comes before fields in the stream I have to buffer the files while awaiting the fields. At least if I need the information from fields in order to process the files.

I guess that email clients which are able to postpone downloading of attachments must rely on a server that parses the email and buffers the files.

Thank you Doug Wilson at https://github.com/pillarjs/multiparty for clarifying this.

Michael
  • 1,764
  • 2
  • 20
  • 36