How do I read csv file in an array in Javascript?

Question

This is my first time with javascript and I am stuck on a problem. I have to read from csv file which contains names of top 1 million websites based on Alexa traffic. The size of csv file is 22.5 MB. Based on some tutorials on net, I am trying to do like this:

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>three.js webgl - interactive cubes</title>
        <meta charset="utf-8">
        <meta name="viewport" content="width=device-width, user-scalable=no, minimum-scale=1.0, maximum-scale=1.0">
        <style>
            body {
                font-family: Monospace;
                background-color: #f0f0f0;
                margin: 0px;
                overflow: hidden;
            }
        </style>
    </head>
    <body>

        <script src="./three.min.js"></script>
        <script src="./jquery-2.1.0.min.js"></script>
        <script src="./stats.min.js"></script>
        <script src="./jquery.csv-0.71.js"></script>

        <script>
        var data = $.csv.toObjects('top_1m.csv');
        console.log(data);
        </script>

    </body>
</html>

But I am not getting any output on console except []. What is wrong with this? Also is the correct way to do this. Basically I need to read file one time in array and then use that array for further processing.

For one, loading, parsing and outputing a 22MB file on a browser is going to take significant time. Is this really something you need to download and do on the client side ? — Prabindh, Feb 15 '14 at 16:36
jquery-csv author here. Browsers have hard limits on memory consumption. jquery-csv parses data synchronously so it probably won't be able to handle 22MB of data. If you're just running reduce on the data to calculate statistics use PapaParse's stream processing. — Evan Plaice, Jan 21 '16 at 11:37

score 3 · Answer 1 · edited May 23 '17 at 11:44

There's a similar question to this here, you must have missed it.

The CSV library parses CSV strings not files. The reason it gives you an empty array
is because it's not complete CSV i.e 1 header + 1 item at the very least.

Here's the documentation in the source code.

LINE 673
    /**
     * $.csv.toObjects(csv)
     * Converts a CSV string to a javascript object.
     * @param {String} csv The string containing the raw CSV data.
     * @param {Object} [options] An object containing user-defined options.
     ...
     **/

As pointed out in the comments, processing a CSV file as huge as that in the browser
is not a wise decision, it's best done at the server.

Here's one way you can open a file and process the contents.
NOTE: It only works in Firefox. Three.js library chokes in IE 8. It complains
about syntax errors(??). You'll get a Uncaught exception: DOMException: NETWORK_ERR
with Opera.

Also, you'll get a syntax error due to parsing invalid(non) XML(i.e the CSV contents)
in Firefox 19.0.2.

It's not the most elegant solution. It just works.

<!DOCTYPE html>
<html lang = "en">
    <head>
        <title>Testing CSV with jQuery csv.0.71</title>
        <meta charset = "utf-8">
        <script src = "./three.min.js"></script>
        <script src = "./jquery-2.1.0.min.js"></script>
        <script src = "./stats.min.js"></script>
        <script src = "./jquery.csv-0.71.js"></script>
        <script>

            function openCSVFile(CSVfileName){
                // create the object
                var httpRequest = new XMLHttpRequest();
                httpRequest.onreadystatechange = function() {
                    processCSVContents(httpRequest);
                }
                // Send the request
                httpRequest.open("POST", CSVfileName, true);
                httpRequest.send(null);
            }

            function processCSVContents(httpRequest){
                console.log("--> here");
                if (httpRequest.readyState == 4){
                    // everything is good, the response is received
                    if ((httpRequest.status == 200)
                    || (httpRequest.status == 0)){
                        // Analys the contents using the stats library
                        // and display the results
                        CSVContents = httpRequest.responseText;
                        console.log($.csv.toObjects(CSVContents));
                        console.log(" => Response status: " + httpRequest.status)
                        console.log(CSVContents);
                    } else {
                    alert(' => There was a problem with the request. ' 
                            + httpRequest.status + httpRequest.responseText);
                    }
                }
            }
        </script>
    </head>
    <body>
        <button type="button" onclick="openCSVFile('pets.csv');">Click Me!</button>
    </body>
</html>

The CSV file, pets.csv contains this:

name,age,weight,species
"fluffy",3,14,"cat"
"vesuvius",6,23,"fish"
"rex",5,34,"dog"

Output:

[{
    name: "fluffy",
    age: "3",
    weight: "14",
    species: "cat"
}, {
    name: "vesuvius",
    age: "6",
    weight: "23",
    species: "fish"
}, {
    name: "rex",
    age: "5",
    weight: "34",
    species: "dog"
}]

There's an alternative way of reading files locally, through HTML5's File API

How do I read csv file in an array in Javascript?

1 Answers1