-1

Essentially, what I have is a form where a user will upload a CSV file. 

<form action="/upload" method="POST" enctype="multipart/form-data">
    <div class="custom-file">
        <input type="file" class="custom-file-input" id="customFile" name="file" required>
        <label class="custom-file-label" for="customFile">Choose file</label>
    </div>
    <input type="submit" class="btn btn-primary btn-block">
</form>

FYI: I'm using getbootstrap.com as my style sheet.

Now, the form sends a POST request to /upload which is where my node.js code is. What I need is for the server to parse this CSV file and extract the data. I've got no idea what to do as all the different NPM packages such as Multer are using a system where the save the file locally and then parse it.

Edit: Forgot to mention this, but temporary local CSV hosting is not an option. I need the client to upload server to process it and push to a DB, can't save the file locally anywhere.

Edit 2: I've used multer and the memory processing option and have gotten the following.

const express = require("express");
const app = express();
var bodyParser = require("body-parser");
var multer  = require('multer');
var storage = multer.memoryStorage();
var upload = multer({ storage: storage });
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.use(express.static("public"));

app.get("/", function(req, res) {
  response.sendFile(__dirname + "/views/index.html");
});
app.post("/upload", upload.single('file'), async (req, res) => {    
  res.send(req.file)
});
const listener = app.listen(process.env.PORT, function() {
  console.log("Your app is listening on port " + listener.address().port);
});

Now once I upload the file, I'm getting the following response (this is what req.file is).

{"fieldname":"file","originalname":"tech.csv","encoding":"7bit","mimetype":"application/octet-stream","buffer":{"type":"Buffer","data":[67,76,65,83,83,32,73,68,44,84,69,67,72,32,35,44,70,73,82,83,84,32,78,65,77,69,44,76,65,83,84,32,78,65,77,69,13,10,54,79,44,54,79,48,49,44,65,110,105,115,104,44,65,110,110,101]},"size":56}

So It's pretty clear that our data happens to be 

67,76,65,83,83,32,73,68,44,84,69,67,72,32,35,44,70,73,82,83,84,32,78,65,77,69,44,76,65,83,84,32,78,65,77,69,13,10,54,79,44,54,79,48,49,44,65,110,105,115,104,44,65,110,110,101

but how do I process that? The csv file data was 

CLASS ID,TECH #,FIRST NAME,LAST NAME
6O,6O01,Anish,Anne

So how do I go from the information provided in the buffer data attribute to the actual data?

Anish Anne
  • 193
  • 3
  • 15
  • It literally took me less than a minute to find a library for parsing CVS from data instead of files, by searching for "npm parse cvs data in memory" then clicking on the _a Stackoverflow question_ and following that to https://github.com/C2FO/fast-csv - So I'm pretty sure this is effectively a duplicate of [Library Recommendations: NodeJs reading csv file](https://stackoverflow.com/questions/23080413/library-recommendations-nodejs-reading-csv-file) – Mike 'Pomax' Kamermans Feb 14 '20 at 04:19
  • @Mike'Pomax'Kamermans What I'm looking for is a way for the user to upload a file and it to be instantly processed, no storing/saving the whole thing on the server anywhere. Directly processed and stored in a DB. – Anish Anne Feb 14 '20 at 12:17
  • @AnishAnne — I just spent 30 seconds skimming the Multer docs before I found the section on using MemoryStorage instead of DiskStorage. – Quentin Feb 14 '20 at 12:21
  • @AnishAnne ...yes? That's what all these libraries can already do, if you read their documentation. They can run "from file" or "from data in a variable in memory". So either your question is "how do I make an upload only end up in memory" (to which the answer is "you absolutely don't, unless you like your server taken down. You REALLY want temp files") or your question is "how do I parse CVS from memory" in which case: these libraries. – Mike 'Pomax' Kamermans Feb 14 '20 at 22:51
  • Wierd, I must have missed that. I'll take a look one more time. – Anish Anne Feb 14 '20 at 23:59

1 Answers1

4

All you have to use is Multer. Multer will return a buffer object which you can then make a string with the .toString() attribute.

Code:

const express = require("express");
const app = express();
var bodyParser = require("body-parser");
var multer  = require('multer');
var storage = multer.memoryStorage();
var upload = multer({ storage: storage });
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.use(express.static("public"));

app.get("/", function(request, response) {
  response.sendFile(__dirname + "/views/index.html");
});

app.post("/upload", upload.single('file'), async (req, res) => {    
  var b = req.file["buffer"]
  console.log(b.toString())
  res.send(b.toString())
});

const listener = app.listen(process.env.PORT, function() {
  console.log("Your app is listening on port " + listener.address().port);
});
Anish Anne
  • 193
  • 3
  • 15