32

I'm processing an old database php array to a new database. The data .php files are in total around 220 MB large.

I've inserted these lines in the script so that it should run fine:

ini_set("memory_limit", "-1");
set_time_limit(0);

This is how I include the products:

// The exports made by PHPMYADMIN, exported as PHP-Array
require_once 'export/tx_ttproductsv2_products.php';
require_once 'export/tx_ttproductsv2_keyword.php';
require_once 'export/tx_ttproductsv2_keywords_in_products.php';
require_once 'export/tx_ttproductsv2_typebook.php';
require_once 'export/tx_ttproductsv2_typegospel7.php';
require_once 'export/tx_ttproductsv2_typemedia.php';

When the script is trying to require them I get this error:

PHP Fatal error: Out of memory (allocated 880541696) (tried to allocate 469762048 bytes) in ......

I've got an local EasyPHP installation running on x64 Win 7 SP1. I've got 6 GB memory, Intel i5.

How can I get PHP to run the whole script without the memory error?

Breeze
  • 2,010
  • 2
  • 32
  • 43
Mathlight
  • 6,436
  • 17
  • 62
  • 107
  • Try to increase `upload_max_filesize` & `post_max_size` ? – Khawer Zeshan Jun 19 '13 at 09:04
  • @KhawerZeshan, i don't upload things? Or am i seeing that wrong? – Mathlight Jun 19 '13 at 09:08
  • exports made by PHPMYADMIN is a file or what? – Khawer Zeshan Jun 19 '13 at 09:10
  • 2
    You should change the memory_limit in the php.ini file. You are however allready using 839.75 MegaByte. That is a lot... If the database is only 220MB large this means your script is creating huge amounts of overhead. Try procesing 1 file at a time instead of all at once – Pinoniq Jun 19 '13 at 09:11
  • @KhawerZeshan, it's an file wich holds an PHP array with the colums... – Mathlight Jun 19 '13 at 09:21
  • @Pinoniq, does it make's an difrnce between the ini_set and the php.ini then? And i was hoping that it wasn't needed to do 1 step at a time... – Mathlight Jun 19 '13 at 09:23
  • 1
    @Mathlight Some configurations don't allow you to overwrite php.ini settings using ini_set. I'm not familiar with easyPHP or Windows so I'm not sure that my answer will work. You are however still using a lot of memory so instead of parsing all files at once, parse them one by one – Pinoniq Jun 19 '13 at 09:31
  • @Pinoniq, I've used it before with less files, and that worked ( ran out of memory, added those lines and it was done ). And it is indeed an large overkill.. So i will parse them 1 by 1 them... A shame, but that's the reality... – Mathlight Jun 19 '13 at 09:33

1 Answers1

21

Memory limitation comes from the OS, not from PHP itself.

Your script allocated 800MB and is trying to allocate further 500MB

Try to run the script on 64bit OS with 64 bit PHP.

Michal M.
  • 1,072
  • 1
  • 11
  • 14
  • That would probably fix a lot, but i'm not able to test it now ( i need it working right away ). – Mathlight Jun 19 '13 at 09:52
  • 7
    The best solution is for you to fetch data and process it in chunks instead of loading all into the memory and then processing. – Michal M. Jun 19 '13 at 10:00
  • I know, I've splitted it up in 3 parts ( coding right now ), and that should work out fine to... Thanks anywhay for the help and ideas ;) – Mathlight Jun 19 '13 at 10:03