How to read a big data (source file) line by line in PHP?

Asked Mar 01 '15 at 12:35

Active Mar 01 '15 at 12:35

Viewed 61 times

I am C++ programmer and trying my hand in PHP and found it is quite similar. I am trying to read a big source file using following code.

$filename = "sample.txt";    /* Having millions of records*/
$file_handle = fopen($filename, "r");

while (!feof($file_handle)) {
       $line = fgets($file_handle);
  /* do some parsing on $line*/  
} 
fclose($file_handle);

But i found when number of records in input file is more than 1 million it crahes and there is no output. How can i fix or optimize the code?

asked Mar 01 '15 at 12:35

Manish

3,341
15
52
87

see this question http://stackoverflow.com/questions/162176/reading-very-large-files-in-php – Gaël Barbin Mar 01 '15 at 12:38
Is there line breaks in your file? May be the first of fgets have to read the entire file. – Gaël Barbin Mar 01 '15 at 12:42
On which line crashes your script? – Gaël Barbin Mar 01 '15 at 12:43
I am browsing records so I am not sure at which line it is crashing but it crashes when input file is large. – Manish Mar 01 '15 at 13:14
Ok, but it crashes on fopen or after being entered in the loop? – Gaël Barbin Mar 01 '15 at 13:30
After entering in the loop – Manish Mar 01 '15 at 13:33
You can add a line size limit to see if it is the problem: fgets($file_handle, $line_size_limit); – Gaël Barbin Mar 01 '15 at 13:46
How much it should be? I am using 4096 is it too much or less? – Manish Mar 01 '15 at 13:48
make tests, try to determine from which size it crashes, and before that, if it is because there a too big line that your script crashes – Gaël Barbin Mar 01 '15 at 13:57
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/71983/discussion-between-gael-and-user15662). – Gaël Barbin Mar 01 '15 at 13:58
try to see some error log, it can be a memory limit, a timeout limit, some error in your parser. try remove your parse code and execute again to see if it's about the parser. – Édipo Costa Rebouças Mar 03 '15 at 12:02
I am planning to increase RAM. After increasing RAM i will dig out further. Thanks – Manish Mar 03 '15 at 14:03

How to read a big data (source file) line by line in PHP?

0 Answers0