-3

Possible Duplicate:
Import CSV file directly into MySQL

I have tried to read data from csv files or comma separated files of huge file size (more than 250MB) in php with different methods (one of these is as under), so i can insert required data from it to mysql database but the program crashes every time before the completion of reading of file. I also tried to increase the memory limit in php.ini. The following errors are faced:-

1.----------------------------------------------

memory_limit = 128M Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 63 bytes) in C:\wamp\www\site\files\plugin\agoda\agoda.plugin.php on line 19

2.----------------------------------------------

memory_limit = 1000M Fatal error: Allowed memory size of 1048576000 bytes exhausted (tried to allocate 1299 bytes) in C:\wamp\www\site\files\plugin\agoda\agoda.plugin.php on line 19

3.----------------------------------------------

memory_limit = 2000M

Null

I need a way to read such files in php without crashing my program.

Community
  • 1
  • 1
adnan masood
  • 6,015
  • 2
  • 13
  • 6
  • 1
    Don't try to read in the entire csv file in one go, and then process it; read a line at a time, process, then read the next line (repeat until eof) – Mark Baker Nov 05 '12 at 10:55
  • if you dont do any computations on the CSV file with PHP consider importing it directly with MySql. Apart from that, there is no question here, because you only state the obvious memory errors which have been asked and answered before. – Gordon Nov 05 '12 at 11:05
  • @Mark Baker, can you please guide me how to read csv line by line and after applying some filter/function/rectifier add it to mysql – adnan masood Nov 06 '12 at 05:59
  • @Gordon, can you please guide me how to read csv line by line and after applying some filter/function/rectifier add it to mysql – adnan masood Nov 06 '12 at 05:59
  • @adnanmasood yes, I can. But please do research first. – Gordon Nov 06 '12 at 09:19

1 Answers1

0

I provide this code modify it to meet your desire 

$source = fopen('zipcode.csv', 'r') or die("Problem open file");
    while (($data = fgetcsv($source, 1000, ",")) !== FALSE)
    {
        $zip = $data[0];
        $city = $data[1];
        $state = $data[2];
        $latitude = $data[3];
        $longitude = $data[4];
        $timezone = $data[5];
        $dst = $data[6];

        mysql_query("INSERT INTO `latlong` (`zip`,`city`,`state`,`latitude`,`longitude`,`timezone`,`dst`) VALUES (".$zip.",'".$city."','".$state."','".$latitude."','".$longitude."',".$timezone.",".$dst.")");

    }
    fclose($source);
Sohail Ahmed
  • 1,667
  • 14
  • 23
  • Don't you have to unset the $data at the end of the while (inside the loop still)? At least with php 4 I had extremely bad experiences there that sometimes the $data was not overwritten as expected but instead the old $data was left as memory corpse (memory leak) and a new $data was created. – Thomas Nov 05 '12 at 10:59
  • No, you should not need to worry about memory leaks... PHP has progressed a bit in the last 15 years – Mark Baker Nov 06 '12 at 08:52