0

I have huge text file and I am trying to read and insert this line by line. this is txt file data.

'REG','KOIL','Kohinoor Industries Ltd.','READY',4.82,2.82,3.82
'REG','EPQL','Engro Powergen Qadirpur Ltd.','READY',36.9495,33.4305,35.19

Function for insert data

            $file_path =FCPATH.'uploads/text/'.$file_name;
            $psx_date=$this->input->post('file_date');
            $open=fopen($file_path,"r");
            $i=1;
            while(!feof($open)){
                $line=fgets($open);
                if($i>2){
                    $values = explode(",",$line);
                    $psx_symbol=str_replace('\'',null,$values[1]);
                    $no_of_rows=read_psx_where($psx_symbol,$psx_date);
                    if($no_of_rows<=0){
                        $psx_data=array(
                            'PSX_SYMBOL'    => $psx_symbol,
                            'PSX_DATE'      => $psx_date,
                            'PSX_HIGH'      => $values[4],
                            'PSX_LOW'       => $values[5],
                            'PSX_CLOSE'     => $values[6],
                            'PSX_DATETIME'  => date('Y-m-d H:i:s'),
                            'PSX_SATUS'     => 1
                        );
                       insert_psx_data($psx_data);
                    }
                }
                $i++;
            }
            fclose($open);

I am first skip first two lines of test file and then I am checking if same symbol is already exist so then skip this line.

This method is working but too much slowdown and exceeding max exectution time.

Bilal Ali
  • 1
  • 1
  • 3
  • its working but too much slowly and that's why I can't insert complete file and exceeding max execution time. – Bilal Ali May 02 '18 at 09:55
  • Run it as a PHP CLI i.e. from the command line – RiggsFolly May 02 '18 at 09:57
  • Try batch insert in CI.It creates batch array and insert data at once – user7596840 May 02 '18 at 09:58
  • Take a look at [this answer](https://stackoverflow.com/a/29751415/1397220), it uses `mysqlimport` to import complete file. Did go a lot faster for my implementation back then, but I had multiple files so did not use it myself. – Brainfeeder May 02 '18 at 10:01
  • I have multiple text files and I am checking line by line if entry is already exist in database so then skip this line that's why I can't use insert_batch. – Bilal Ali May 02 '18 at 10:08
  • any one can have another way to insert text file data line by line in mysql so suggest me. – Bilal Ali May 02 '18 at 10:15
  • Run it from Command line, if you run one by one from code. it will take much execution time. – Gautam D May 02 '18 at 10:19
  • @GautamD 500 to 600 text files which data we have to insert in database so i have to generate funtion for import that files data. – Bilal Ali May 02 '18 at 10:24

0 Answers0