D3: ignore certain row in .csv

Question

I have a .csv file that looks like:

The Start Date, The User Id, The User Name,
date, id, name,
12-12-12, 12345, John Doe
01-02-13, 67891, Jane Doe

The first row are descriptions for the keys on the second row. I'm loading in the .csv file as follows:

d3.csv("data.csv", function(error, data) {
  data.forEach(function(d) {
    d.date = parseDate(d.date);
    d.id   = formateID(d.id);
    d.name = formatename(d.name);
    d.close = +d.close;
  });

I want to keep the first row (description) separate from my data model because I still plan on using it down the road, however I'm not allowed to modify the format of the .csv file. Is there a way I can only capture this information:

date, id, name,
12-12-12, 12345, John Doe
01-02-13, 67891, Jane Doe

from the csv?

[This question](http://stackoverflow.com/questions/2003815/how-to-remove-first-element-of-an-array-in-javascript) should help. — Lars Kotthoff, Jun 13 '13 at 15:12
@LarsKotthoff There's an extra row of headers in the CSV file, and I don't see how splicing would do anything here. Maybe I'm missing something(?). — meetamit, Jun 13 '13 at 16:10
`d3.csv` returns an array, which you are iterating over (using `forEach`). Each element of the array is a row. So to get rid of the first row, simply remove the first element of the array. — Lars Kotthoff, Jun 13 '13 at 17:28
@LarsKotthoff But the thing that's returned by `d3.csv` is an array-of-objects, not an array-of-arrays, and each of those objects would have the keys of the 1st row of headers ('The Start Date', 'The User Id', etc) rather than the 2nd row of headers, which I think is what the OP wants. — meetamit, Jun 13 '13 at 18:13
Ah ok, that would make sense. Although even then the approach would work as the loop afterwards is reformatting anyway. — Lars Kotthoff, Jun 13 '13 at 18:42

meetamit · Accepted Answer · 2016-02-06T08:48:42.150

The d3.csv() function does both the loading and the parsing without giving you a chance to intervene. Since you need to intervene, you'll want to load the csv as plain text –– without parsing it –– then remove the first line, and then pass it as a String into the csv module for parsing.

For loading, use d3.xhr() For parsing, use d3.csv.parse().

You should end up with something like this:

d3.xhr('data.csv').get(function (err, response) {
  var dirtyCSV = response.responseText;
  var cleanCSV = dirtyCSV.split('\n').slice(1).join('\n');
  var parsedCSV = d3.csv.parse(cleanCSV);
})

Edit

Here is a less costly transformation than the one above (this has not been tested/debugged):

d3.xhr('data.csv').get(function (err, response) {
  var dirtyCSV = response.responseText;
  var firstEOL = dirtyCSV.indexOf('\n');
  var parsedCSV = d3.csv.parse(cleanCSV.substring(firstEOL+1));
})

score 0 · Answer 2 · answered Oct 14 '19 at 09:54

For those who want to ignore certain rows from csv, following snippet can be used.

//Read the data
d3.csv("abc.csv?"+Math.random(),
  function(d){
    //if current row's value column is blank, then ignore it. Otherwise do further process.
    if(d.value!=''){ 
      return { date : d3.timeParse("%Y-%m-%d %H:%M:%S")(d.date), value : d.value }
    }
  },

D3: ignore certain row in .csv

2 Answers2

Edit

Linked