-2

i am building a script to migrate data from a dbase file to a mysql table, but with some files (large ones) it gives me this error:

Fatal error: Out of memory (allocated -1678508032) (tried to allocate 64 bytes) in D:\wamp\www\dbf2MySql\LeerDbase.php on line 65

This is the code 

echo "Por favor espere...<br/>";
for ($valor = 1; $valor <= count($archivos); $valor++) {
    //echo $valor; 
    //echo $archivos[$valor]; echo "<br/>";
    $fichero_dbf = "tmp/". $archivos[$valor];
    $conex       = dbase_open($fichero_dbf, 0); //abro el archivo
    if($conex){ //si la conexión es buena, realizo la migración
        $arrData = array();
        $total_registros = dbase_numrecords($conex);
        for ($i = 1; $i <= $total_registros; $i++){
            $arrData[] = dbase_get_record($conex,$i); //<-- Here is the memory error line
        }
        // Obtener la información de columna
        $info_columna = dbase_get_header_info($conex);
        //Busco los titulos o cabeceras

        for ($i = 0; $i < count($info_columna); $i++) {
            $titulos[$i] = $info_columna[$i]['name'];
            if (in_array($info_columna[$i]['name'], $titulos)) {
                $titulos[$i] = $info_columna[$i]['name']."_".$i;
            }
        }
        //print_r($titulos); 
        //echo DibujarTabla($titulos, $arrData);
        $arreglo = explode(".", $archivos[$valor]);         
        //Paso los datos a mysql
        CrearTablaMySql($arreglo[0], $dbf2MySql, $titulos, $arrData, $database_dbf2MySql, $total_registros);

    }else{
        echo "No se pudo acceder al fichero dbf -> '" . $fichero_dbf . "'<br/>"; //Si me da un error muestro un error
    }
    dbase_close($conex); //Cierro la conexión al archivo
    if ($valor == (count($archivos))) {
        echo "FINALIZADO";
        mysqli_close($dbf2MySql);
    }
}

Any help would be appreciated, let me know if you need more info to help me solve this issue

EDITED

http://goo.gl/aHURGl

UPDATE

I use now command line, but i get this error now using php.exe

D:\wamp\bin\php\php5.5.12>php.exe -f "D:\wamp\www\dbf2MySql\CommandScript.php" errmxite.dbf Por favor espere...

Fatal error: Call to undefined function dbase_open() in D:\wamp\www\dbf2MySql\Co mmandScript.php on line 33

Call Stack: 0.0007 273304 1. {main}() D:\wamp\www\dbf2MySql\CommandScript.php:0

Here is the updated code

http://goo.gl/drf6Ht

SOLUTION

I used php command line to execute the scripts with some changes, and now it works without issues, thanks to all

user3282377
  • 63
  • 1
  • 9
  • possible duplicate of [PHP Fatal error: Out of memory (allocated 80740352) (tried to allocate 12352 bytes) in](http://stackoverflow.com/questions/6314733/php-fatal-error-out-of-memory-allocated-80740352-tried-to-allocate-12352-byt) – Sverri M. Olsen Oct 23 '14 at 20:53
  • 1
    Don't read all the records from dbase and put them in an array before writing them to MySQL... read one, write one, read next, write next, etc – Mark Baker Oct 23 '14 at 20:55
  • Or save sql queries into the file and then import into MySQL. Do not store everything in the memory. – Cheery Oct 23 '14 at 20:59
  • The script is to migrate, i build the mysql statements to later execute them – user3282377 Oct 23 '14 at 22:40
  • It gives the same error when i execute one per cicle @Cheery – user3282377 Oct 23 '14 at 23:03
  • @user3282377 Without seeing the new code we can not give any advises. It could happen that you have huge amount of data in a single row. – Cheery Oct 23 '14 at 23:07
  • @Cheery that's the case, i am thinking in a way to split that dbf file, it has more that 500 Mb on data, just the titles (headers) are more that 400 – user3282377 Oct 24 '14 at 00:04
  • @user3282377 What can I tell you. Try to use command-line utilities to get the data, not by php. Form sql queries and import them into MySQL. – Cheery Oct 26 '14 at 20:15
  • @Cheery i updated my question now, i use command line, but it seems that it has issues with the dll on command lines, do i have to do something to register the dll with php.exe (i already did on php.ini)? – user3282377 Oct 26 '14 at 21:33
  • @user3282377 no, with command line I meant other utilities, not by php. Do you need to do it this conversion only once or from time to time? If it is a one-time job - do it not by php. Probably php reads the whole file into memory (never used dbase). The idea with setting memory usage is good, if you know the limits - amount of memory installed and needed to parse the database. Do not set it to -1, set a few gigs (if you have this amount of memory on your pc) – Cheery Oct 26 '14 at 21:55
  • @Cheery I did that and the pc freezes xD, now i solved ussing command line of php without issues – user3282377 Oct 30 '14 at 14:27

1 Answers1

0

You can try to increase the allocated memory in your php.ini 

memory_limit = 1024M  #or whatever size you wanna allocate to php scripts

or if you want to execute the script without limit you can try adding this code 

ini_set('memory_limit', '-1');

on the beginning of your file.

diokey
  • 176
  • 1
  • 2
  • 12
  • It crashes the pc, but thanks anyway, still looking for a better solution, the file have more that 6 million records – user3282377 Oct 26 '14 at 21:34