Upload CSV to Heroku via AJAX - Node/Express

Question

I have a database I would like users to be able to update using CSV files. I'm trying to use the Postgres COPY function to do this but haven't gotten it to work.

HTML:

<h2>Mass Update</h2>
  <form enctype="multipart/form-data" id="mass">
    <input type="file" id="file" name="upload"/><br/>
    Replace <input type="checkbox" name="replace"/>
  </form>
<button id="trigger-mass">Submit</button>

jQuery

$('#trigger-mass').on('click', function(){
var fd = new FormData(document.getElementById('mass'));
$.ajax({
    url: base + table,
    processData:false,
    data:fd,
    type:'POST'
});

Node

index.js

 app.use('/', express.static(__dirname + '/static'));
 app.use(express.bodyParser({uploadDir:__dirname +'/static/tmp', keepExtensions:true}));

This may not seem best practice, but it is ok in my case as I want these uploads to be available for download. I was receiving Error:ENOENT open"..." if the tmp dir was outside the static dir.

routes.js

 // Clear table and refill with data
 app.post('/data/:table', auth, function(req, res) {
    var oMsg = quickRes(res, req);
    console.log('REQ DATA')
    console.log(req.body);
    console.log(req.files);
...// Here is where I would issue the COPY command through pg
 });

Unfortunately, I haven't had anything show up in req.files. I've also tried just directly uploading the file in AJAX as data:document.getElementById('file').files[0] to no avail. Any suggestions?

score 2 · Accepted Answer · answered Nov 22 '13 at 08:02

2

You need to explicitly set contentType to false in $.ajax():

$.ajax({
  url         : base + table,
  processData : false,
  contentType : false, // <-- required
  data        : fd,
  type        : 'POST'
});

Otherwise, jQuery will set it to application/x-www-form-urlencoded; charset=UTF-8, which isn't appropriate for file uploads.

answered Nov 22 '13 at 08:02

robertklep

198,204
35
394
381

This was one part of the problem. I was also getting the `Error: ENOENT open "..."` when bodyParser was trying to store the file in `tmp`. I added the `tmp` directory beneath the `static` directory and it worked. So this was also a permissions issue of my file system. Thanks much. – Eric H. Nov 22 '13 at 17:23
Also, [this answer](http://stackoverflow.com/a/8758614/1149459) may be helpful to others. – Eric H. Nov 22 '13 at 17:28

DF_ · Answer 2 · 2013-11-22T08:10:00.773

1

Could we get a look at the middleware you've included and what version of express you're using? express.multipart will be deprecated in the next version so you have to use an alternative solution. The general consensus thus far is to use the connect-busboy npm package as a middleware for file uploads.

Example index.js:

app.use(express.json()); // These replace the bodyParse middleware due to the tmp folder vulnerability
app.use(express.urlencoded());
app.use(busboy());

app.post('/data/:table', auth, function (req, res) {
    var fstream;
    req.pipe(req.busboy);
    req.busboy.on('file', function (fieldname, file, filename) {
        fstream = fs.createWriteStream(__dirname + '/tmp/' + filename);
        file.pipe(fstream);
        fstream.on('close', function () {
            res.redirect('/'); // Or wherever you want to go next
        });
    });
});

Included node packages would be express, fs, and connect-busboy. See this Github Issue to confirm the deprecation of multipart.

edited Nov 22 '13 at 08:10

answered Nov 22 '13 at 07:50

DF_

3,743
25
34

That issue doesn't say `bodyParser` will be deprecated, it says [that the `multipart` part of `bodyParser` will be removed from it](https://github.com/visionmedia/express/issues/1793#issuecomment-27016607). – robertklep Nov 22 '13 at 08:04
I'm on Express 3.1 and did get it to work using the above suggestion. Good chance I never upgrade bc this is a short term API I'm building. But does this mean that all form uploads to express will have to use another plugin in the future? – Eric H. Nov 22 '13 at 17:21
Only for multipart. I guess connect could build in busboy to bodyParser, but it's up to them. – DF_ Nov 22 '13 at 18:11
@Deif I think `express.multipart` will still exist, just as a separate middleware instead of being part of `bodyParser` (so you have to be more explicit if you want your server to support file uploads). – robertklep Nov 22 '13 at 18:17
Yup, but at the moment it references the connect.multipart middleware directly without any other code, so I meant they would either have to wait for connect to update it or write it themselves. – DF_ Nov 22 '13 at 18:43

Upload CSV to Heroku via AJAX - Node/Express

2 Answers2