reading in csv with date parsing

Question

Having a very hard time loading in my data correctly. Here is the header row with the first content row:

BookingID,Type,Status,Unit Booked,Unit Owner,Destination,Booking Date,Checkin,Checkout,Renter - FirstName,Renter - LastName,Renter - EmailAddress,Renter - WorkPhone,Renter - HomePhone,#Adults,#Children,Total Stay,Total Paid,Total Due
15642889,House,Confirmed,GV T3 #2106,,,3/20/2016 7:00:00 PM,3/23/2016 3:00:00 PM,3/28/2016 11:00:00 AM,FirstName,LastName,first&last@gmail.com,+1 (000) 000-0000,+1 (000) 000-0000,2,0,895,895,0

And the relevant lines of loading in and parsing my csv as I want:

var parseDate = d3.time.format("%m/%d/%Y %H:%M:%S %p").parse;
var data = d3.csv("Sales Export Friendly 3-19-17.csv", function(data) {
    return {
        unit: data["Unit Booked"],
        date: parseDate(data["Booking Date"]).getMonth() + 1,
        checkin: parseDate(data["Checkin"]).getMonth() + 1,
        LOS: parseDate(data["Checkout"]).valueOf() - parseDate(data["Checkin"]).valueOf()/(24*60*60*1000),
        total: +data["Total Stay"],
        avgNight: (+data["Total Stay"]) / ((new Date(data["Checkout"]).valueOf() - new Date(data["Checkin"]).valueOf())/(24*60*60*1000))
        };
});

The idea is that I will then do something like this:

d3.parcoords()("#TopLeft").alpha(0.4)
        .data(data)
        .dimensions(data.columns)

If I try to console.log(data.columns); after my callback function, I get undefined. Here is what console.log(data); prints, which is quite odd looking:

Object { header: Cn/u.header(), mimeType: Cn/u.mimeType(), responseType: Cn/u.responseType(), response: Cn/u.response(), get: Cn/</u[n](), post: Cn/</u[n](), send: Cn/u.send(), abort: Cn/u.abort(), on: M/<(), row: e/o.row() }

And the error codes I am currently getting:

TypeError: data.slice is not a function

When calling data(data) on d3.parcoords and

TypeError: e is undefined

On this line:

checkin: parseDate(data["Checkin"]).getMonth() + 1,

I am terribly confused as to what is going wrong here. I am using d3 v3.

Answer 1

EDIT: I am wrong. The callback is per row. However, the data are indeed loaded asynchronously in the newest version of d3. I had some trouble finding the right way to use this function, but I found the solution. d3.csv now returns a promise. To use the parsed data, pass doSomething to the promise.then().

Original comment, for context:

The CSV file is loaded asynchronously, so setting data=d3.csv(path,callback) makes little sense. When you declare a callback, that is where things must happen. The callback may refer to a global variable, or just use a local variable and pass the result to the next stage. Judging by your code, you want to output an object that holds filtered/postprocessed columns stored under their respective keys.

Also, I doubt that the time parse method can be applied to an array. To do that, you likely have to use array.map.

I would suggest something like this (not tested):

var parseDate = d3.time.format("%m/%d/%Y %H:%M:%S %p").parse;
d3.csv("Sales Export Friendly 3-19-17.csv", function(data) {
    //Note that data is a local in this function scope
    let filtered_data = {
        unit: data["Unit Booked"],
        date: data["Booking Date"].map(s=>parseDate().getMonth()+1),
        checkin: data["Checkin"].map(s=>parseDate().getMonth()+1),
        total: data["Total Stay"].map(s=>+s),
    };

    filtered_data.LOS = [];
    filtered_data.avgNight = [];
    let N = data["Checkout"].length;
    for (let i=0; i < N; i++) {
        filtered_data.LOS[i] = parseDate(data["Checkout"][i]).valueOf() -parseDate(data["Checkin"][i]).valueOf()/(24*60*60*1000),
        filtered_data.avgNight[i] = (+data["Total Stay"][i])/filtered_data.LOS[i];
    }
    doSomething(filtered_data);
});