PHP looping through file internal error without logs

Question

I use AJAX to get some JSON data from a file on my server.

Server-side code

 $file = fopen($path); /*~1M rows and ~650MB file*/

 /*Headers*/
 $cols = fgetcsv($file);
 $cols[0] = 'id';  

 /*Loop through each line*/
 $file_arr = [];
 while (($line = fgetcsv($file)) !== FALSE) {
     $file_arr[] = array_combine($cols,$line);
     /*The code crashes somewhere during this loop*/
 }
 fclose($file);

/*Returns some JSON*/

However, I get an Internal Error but there's nothing in the log file. I added some Log::Info through out my code and I have managed to find that the error is due to the loop but I cannot figure out what because I do not have log for the internal error.

Are your arrays, `$cols` and `$line` of equal size? In other words is there one element in each because `$cols` has exactly one element. My bet is `$line` has more than one element. This will return FALSE if the number of elements for each array isn't equal. — Jay Blanchard, Dec 17 '15 at 17:41
@JayBlanchard That's what I presume. Plus, I know that the while loop can do multiple iterations without any problems. However, I cannot for sure rule out that one line somewhere might be of a different size — Wistar, Dec 17 '15 at 17:44
You may want to test `$line` before you do any combining and then spit out an exception report for those lines containing more than one element. — Jay Blanchard, Dec 17 '15 at 17:45
No logs, well there's surely something, somewhere. If you can't access logs, then add error reporting to the top of your file(s) right after your opening PHP tag for example ` — Funk Forty Niner, Dec 17 '15 at 17:49
@JayBlanchard something like `if(count($line) <> count($cols)) throw new Exception('Line and cols are not of equal size')`? — Wistar, Dec 17 '15 at 17:50
@Fred-ii- I have tried `ini_set('display_errors', 1);` the results is that instead of throwing `Internal Server Error` I get `Invalid JSON` from my AJAX function (and still no log). — Wistar, Dec 17 '15 at 17:51
PHP will not throw an error on invalid JSON until you attempt to parse it with PHP. Since JavaScript is trying to parse it is returning the error. The invalid JSON *could be* caused by `array_combine()` returning `false` during the loop. Can we see your JSON? — Jay Blanchard, Dec 17 '15 at 17:53
@JayBlanchard I have try to throw an exception or write a log if the array size were no equal. There's still no log and an Internal server error. Actually, it worth noting that my file has around 1M lines and is ~650MB. It could but an execution timeout or a memory exhaustion... — Wistar, Dec 17 '15 at 18:14
Yep. Until you solve the Internal Error you will not be ably to reliably test. — Jay Blanchard, Dec 17 '15 at 18:17
what I suggest you do then, is to test it with a really smaller/cut-down version of your JSON. If it works, you'll know it's probably a timeout issue. You can increase that time/memory. You can also see if you can write the error log to file http://php.net/manual/en/function.error-log.php --- http://stackoverflow.com/questions/3531703/how-to-log-errors-and-warnings-into-a-file — Funk Forty Niner, Dec 17 '15 at 18:19

score -1 · Answer 1 · answered Dec 17 '15 at 18:43

Everything works well when the size of the file is reduced. Therefore, it is probably a memory exhaustion / execution timeout error.

Instead of opening the looping through the whole file at once, I will limit it according to what need be display to the end user only.

PHP looping through file internal error without logs

1 Answers1