Read CSV with CsvReader from url

Question

I have to read CSV file from URL like : http://somedomain.com/getdata.aspx?p1=param1&p2=param2

HttpWebRequest req = (HttpWebRequest)WebRequest.Create(_serviceUrl);
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

StreamReader sr = new StreamReader(resp.GetResponseStream());
CsvReader csvread = new CsvReader(sr, new CsvConfiguration()
    {

    });
List<Entity> record = csvread.GetRecords<Entity>().ToList();
sr.Close();

but i do it with no luck
the header fields goes with <...> like this:

<Field1>;<Field2>;<Field3>

and data

stringValue;777;anotherStringValue

any help?

What exactly means "with no luck"? You might want to have a look at the FileHelpers project: http://filehelpers.sourceforge.net/ which is very powerful in working with CSV files. — Jens H, Jun 19 '13 at 13:07
@JensH IOException{"Unable to read data from the transport connection: The connection is broken."} — Roar, Jun 19 '13 at 13:21
Well, this does not sound like a file parsing problem at all. It seems like your service communication has a problem. — Jens H, Jun 19 '13 at 13:26
my question is not about parsing, it's about all stuff, how can i download file and parse — Roar, Jun 19 '13 at 13:28
Please post the full exception message for more information. — Jens H, Jun 19 '13 at 13:28
{"Unable to read data from the transport connection: The connection was closed."} — Roar, Jun 19 '13 at 13:40
`Fields 'Field' do not exist in the CSV file.` i think this is because of header — Roar, Jun 19 '13 at 13:46
@JensH there is a solution, so we don't need down vote anymore — Roar, Jun 19 '13 at 19:06

Roar · Accepted Answer · 2013-06-30T10:20:05.003

for me, solution is to read a file and map it to entity

    HttpWebRequest req = (HttpWebRequest)WebRequest.Create(_serviceUrl);
    req.KeepAlive = false;
    req.ProtocolVersion = HttpVersion.Version10;
    HttpWebResponse resp = (HttpWebResponse)req.GetResponse();

    using (StreamReader streamReader = new StreamReader(resp.GetResponseStream()))
    {
        CsvConfiguration configuration = new CsvConfiguration()
        {
            Delimiter = ";",
            HasHeaderRecord = true,
            IsHeaderCaseSensitive = false
        };
        configuration.RegisterClassMap<EntityMap>();
        CsvReader csvread = new CsvReader(streamReader, configuration);
        List<Entity> record = csvread.GetRecords<Entity>().ToList();
    }

and there is a mapping:

public class EntityMap : CsvClassMap<Entity>
    {
        public override void CreateMap()
        {
            Map(m => m.Field1).Name("<Field1>");
            Map(m => m.Field2).Name("<Field2>");
        }
    }

score 0 · Answer 2 · edited May 23 '17 at 12:05

It seems like there are multiple problemes here that you are coping with. Unfortunatelly there is very little information from you which exact troubles you are facing and at which points in your application.

As I am not aware of a CsvReader class in the .NET framework so I assume you are looking for an existing framework (or are already using one).

As for the code snippet you postetd, I suggest splitting this up into several pieces. Like, first load the file onto your disk in some temporary location, so you can clean up the request/response objects right after they have finished their work.

I am not sure how the StreamReader behaves when working with that ResponseStream, there might arise some more troubles when accessing sources you possibly do not have sufficient access privileges for.

After you successfully downloaded the file to a meaningful location, you can process it with whatever tool or framework you want to you use.

Depending on the complexity of your CSV file this StackOverflow thread might also offer some inspirations for parsing the file: Very simple C# CSV reader

There should be plenty of internet resources to help you get started with your single steps, for example:

Download the file: Downloading a file via HTTP post and HTTP get in C# This discusses two alternatives using HttpWebRequest to fetch your file and how to save it to your disk; Or Download Files from Web explaining this using the WebClient class.
Parse the CSV file: A very helpful and powerful library: FileHelpers Library Have a look at the documentation and samples to see how this works. The Delimeted File Engine might be just what you need. Alternatively have a look at LINQ to CSV library

"As I am not aware of a CsvReader class in the .NET framework" - [TextFieldParser Class](http://msdn.microsoft.com/en-us/library/microsoft.visualbasic.fileio.textfieldparser.aspx). Very useful, but relatively unknown. Been part of the framework since 2.0. :) — Tim, Jun 19 '13 at 19:15

Read CSV with CsvReader from url

2 Answers2