0

I'm trying to open/read a 6MB CSV file with PHP like so:

$lines = file("/path/to/my/file.csv");

But I'm getting the following error:

PHP Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 6461150 bytes) in /path/to/php_file.php on line 488

As you can see, our PHP memory settings are set to 1GB (dedicated server, not limited by any shared hosting package etc).

My question is should a 6MB CSV file really be using over 1GB when read into a variable (array). I'm just a bit confused because I'm sure I've opened larger CSV files with PHP before without problems on this server.

Shouldn't make a difference but we're using PHP 5.3 on Ubuntu 12.04 Server.

BT643
  • 3,495
  • 5
  • 34
  • 55
  • Is that the only thing the script is doing? Is there a reason you need the full file in memory? – Jim Mar 21 '14 at 10:36
  • The script does do some stuff before (to big to post the entire thing on here), but this is the line it's failing on. I'll have a quick scan through and see if I can spot anything else which may be "wasting" memory. We're looping through the file doing some basic processing on each line. – BT643 Mar 21 '14 at 10:40
  • Calling `http://uk1.php.net/memory_get_usage` before the `file` line may help track it down. – Jim Mar 21 '14 at 10:43
  • Thanks. I unset a few variables which were no longer needed before the call and it's now working fine :) It's still using nearly 1GB of memory somehow before the call according to `memory_get_usage` :/ so need to look into that. But it takes it under the limit enough for this to work. Thanks again! – BT643 Mar 21 '14 at 11:05
  • check my solution at http://stackoverflow.com/a/22744300/2037323 which includes some comparisons also – Raza Ahmed Mar 30 '14 at 13:08

2 Answers2

1

You should process the csv line by line to prevent memory exhaustion. Try something like this:

$f = fopen("my.csv", "r");
if(true === is_resource($f))
{
    while(false === feof($f))
    {
        $line = fgetcsv($f, 8192);
        // do something with the data here
    }
    fclose($f);
}

Using fgetcsv() to parse the csv is also far easier than using file().

daniel.kollorz
  • 68
  • 1
  • 6
-2

Try to restart the apache server.

Arend
  • 3,741
  • 2
  • 27
  • 37
Deepak Mane
  • 105
  • 6
  • It's a production server. I'd really like to avoid kicking off a few hundred users unless I really need to. If you can suggest any way of seeing if this would help beforehand (viewing total used memory, etc) then I'd be interested in looking into that. – BT643 Mar 21 '14 at 10:36