how to use online CSV file data as value without downloading the file

Question

Hi I have a link that is download a CSV file, I would like to use the data inside to build my website, but I don't wont to download the file, is there anyway to do it? the link is: http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download

score 2 · Answer 1 · answered Aug 23 '12 at 20:43

2

You need to get the file to get the data inside it.

Have your server delete it afterwards.

answered Aug 23 '12 at 20:43

databyss

6,318
1
20
24

You can Open a stream to the file via `WebClient.OpenRead()` and assign the results to a Stream (see my answer). This is C#, however. – StoriKnow Aug 23 '12 at 20:54

score 1 · Answer 2 · answered Aug 23 '12 at 20:46

1

You can fetch the contents of the file at runtime using cURL.

$url = "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download";
$ch = curl_init(); 
curl_setopt($ch, CURLOPT_URL, $url); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); 
$data = curl_exec($ch); 
curl_close($ch);

$data will now contain the contents of that file.

answered Aug 23 '12 at 20:46

Bad Wolf

8,206
4
34
44

I just try it and result is "Fatal error: Call to undefined function curl_init()" is there anything I forget to add code: ' Untitled Document ' – Priscilla Ip Aug 23 '12 at 20:54
@PriscillaIp If you're getting an error saying that `curl_init()` is undefined it means that the PHP cURL library is not installed on your server. If you can't/don't want to install it try using Stegrex's `fopen()` solution instead. – Bad Wolf Aug 23 '12 at 20:57
Do you know any good instruction for installing the cURL library? – Priscilla Ip Aug 24 '12 at 16:25
@PriscillaIp Priscilla, please read [this](http://stackoverflow.com/questions/1347146/how-to-enable-curl-in-php) and [this](http://stackoverflow.com/questions/2939820/how-to-enable-curl-installed-ubuntu-lamp-stack). They are instructions on how to enable cURL (if installed) and how to install cURL. Also check out the [cURL installation docs](http://www.php.net/manual/en/curl.installation.php) from the PHP manual. – Stegrex Aug 27 '12 at 04:41

Stegrex · Accepted Answer · 2012-08-29T19:10:10.723

0

Use fopen() or PHP cURL library:

fopen() (Usually this is not as safe as cURL, and you might run into issues when fopen is not allowed in the PHP settings file, but it is a quick and dirty way to do it):

// Load file into resource from URL.
$fileHandle = fopen("http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download", "r");
// Edit: Load file resource into string.
$fileContents = fread($fileHandle);
echo $fileContents;

cURL (The preferred way. Read up on cURL here: http://php.net/manual/en/book.curl.php):

$curl = curl_init(); // Open a cURL library resource.

// Set the resource option for the URL.
curl_setopt($curl, CURLOPT_URL, "http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download");
// Set the resource option to let it return a value from the execution function.
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);

$file = curl_exec($curl); // Execute cURL, load the result into variable $file.
curl_close($curl); // Close the cURL resource.

print_r($file); // The file should now be a string, exactly the CSV string that was scraped.

edited Aug 29 '12 at 19:10

answered Aug 23 '12 at 20:43

Stegrex

4,004
1
17
19

1

I have try the fopen, with echo, but I get "Resource id #3" only. Is there anything i have missed? – Priscilla Ip Aug 29 '12 at 17:21
@PriscillaIp Sorry for the confusion. Function `fopen()` returns a file resource, which you must manipulate in order to get the string. Use `fread()` as in my edited answer and echo that return value, not the resource. If you can, I would highly recommend installing cURL as you have more control over security, etc. – Stegrex Aug 29 '12 at 19:11
Thanks Stegrex, for the fread(file,length). I also would like to use cURL, but the library install fail in my computer, it is painful and i already run out of time. – Priscilla Ip Aug 31 '12 at 15:24
@PriscillaIp Reinstalling PHP with cURL can be a pain, I agree. Hopefully the fread() solution solved what you needed. – Stegrex Aug 31 '12 at 15:44

score 0 · Answer 4 · answered Aug 23 '12 at 20:47

Are you using C# or PHP? My solution is in C#.

HttpWebRequest req = (HttpWebRequest)WebRequest.Create("http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download");
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
StreamReader reader = new StreamReader(resp.GetResponseStream());

string CSVContents = reader.ReadToEnd();

CSVContents now contains the contents of the file.

StoriKnow · Answer 5 · 2012-08-24T13:01:48.463

0

This is a way to read the data via a Stream rather than downloading the contents of the data to your physical disk. This is in C# (couldn't tell if you wanted PHP or C# since your tags include both)

string uriString = @"http://www.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=nasdaq&render=download";
WebClient myWebClient = new WebClient();
Stream myStream = myWebClient.OpenRead(uriString);    //open a stream to the resource
StreamReader sr = new StreamReader(myStream);
Console.WriteLine(sr.ReadToEnd());                    //print contents of stream

myStream.Close();

edited Aug 24 '12 at 13:01

answered Aug 23 '12 at 20:52

StoriKnow

5,738
6
37
46

*"without actually downloading the file"* ... that's a bit misleading. You still have to bring the data from there to here. Just because it's not all buffered in memory at the same time doesn't mean you don't download the file to get the data it contains. – Aug 23 '12 at 22:14
@rdlowrey thanks for the comment, I've updated the answer to be (hopefully) clearer. – StoriKnow Aug 24 '12 at 13:02

how to use online CSV file data as value without downloading the file

5 Answers5