how to read a csv file from a url?

Question

Im trying to create a web service which gets to a URL e.g. www.domain.co.uk/prices.csv and then reads the csv file. Is this possible and how? Ideally without downloading the csv file?

Without downloading the CSV file? How do you expect to read it? Or do you just mean read a portion of it without having to download the whole thing first. — Tim S., Jun 18 '12 at 11:44
Downloading doesn't mean you have to save the file to disk if that is what you think. That being said, you can't read the file without downloading it. — Botz3000, Jun 18 '12 at 11:44

score 36 · Accepted Answer · edited Jul 31 '15 at 13:29

36

You could use:

public string GetCSV(string url)
{
    HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
    HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

    StreamReader sr = new StreamReader(resp.GetResponseStream());
    string results = sr.ReadToEnd();
    sr.Close();

    return results;
}

And then to split it:

public static void SplitCSV()
{
    List<string> splitted = new List<string>();
    string fileList = getCSV("http://www.google.com");
    string[] tempStr;

    tempStr = fileList.Split(',');

    foreach (string item in tempStr)
    {
        if (!string.IsNullOrWhiteSpace(item))
        {
            splitted.Add(item);
        }
    }
}

Though there are plenty of CSV parsers out there and i would advise against rolling your own. FileHelpers is a good one.

edited Jul 31 '15 at 13:29

Marcos Meli

3,468
24
29

answered Jun 18 '12 at 11:52

dtsg

4,408
4
30
40

1

If you're reading the whole file into a string anyway and do not need the additional flexibility of `WebRequest` you can just use `WebClient.DownloadString`. – Joey Jun 18 '12 at 12:14
1

If you use `VisualBasic.IO.TextFieldParser` you can use something that's already built into .NET by just adding a reference to `Microsoft.VisualBasic`. After that use this instead of the stream reader. `using (TextFieldParser parser = new TextFieldParser(response.GetResponseStream()))` – B Minster Jul 01 '20 at 16:57

score 2 · Answer 2 · answered Jun 18 '12 at 11:45

In your Web Service you could use the WebClient class to download the file, something like this ( I have not put any exception handling, not any using or Close/Dispose calls, just wanted to give the idea you can use and refine/improve... )

using System.Net;

WebClient webClient = new WebClient();
webClient.DownloadFile("http://www.domain.co.uk/prices.csv");

then you can do anything you like with it once the file content is available in the execution flow of your service.

if you have to return it to the client as return value of the web service call you can either return a DataSet or any other data structure you prefer.

hmmmm so if i wanted to put the csv into a datatable, would i do datatable table = webClient — Beginner, Jun 18 '12 at 13:14

score 1 · Answer 3 · edited Dec 30 '19 at 09:51

Sebastien Lorion's CSV Reader has a constructor that takes a Stream.

If you decided to use this, your example would become:

void GetCSVFromRemoteUrl(string url)
{
    HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
    HttpWebResponse response = request.GetResponse() as HttpWebResponse;

    using (CsvReader csvReader = new CsvReader(response.GetResponseStream(), true))
    {
        int fieldCount = csvReader.FieldCount;
        string[] headers = csvReader.GetFieldHeaders();

        while (csvReader.ReadNextRecord())
        {
            //Do work with CSV file data here
        }
    }

}

The ever popular FileHelpers also allows you to read directly from a stream.

score 0 · Answer 4 · answered Jun 18 '12 at 11:48

0

The documentation for WebRequest has an example that uses streams. Using a stream allows you to parse the document without storing it all in memory

answered Jun 18 '12 at 11:48

miniBill

1,743
17
41

how to read a csv file from a url?

4 Answers4

Linked

Related