PHP opening a large file

Question

What's the best way to read a .TXT file (The file size is 225mb). I want to open the file and loop thru it and find information via REGEX.

Example data in the file :

00424333060001410100100BILLLLOYD BRRUSSELL & 12675 MAKALISO AVE WEST WORKS TOWN KS 23456-1035 3341310350630200500004200000001887800001789IWD QM1214200400003367250001799900001287IWD QM 000000000000000000000000000000 000000000000000000000000000000

The problem I am having is the file taking forever to open. And to search thru takes a while. My loop could have 75 items I need to search.

    $name2 = "BILLLLOYD BRRUSSELL ";
    $RE21 = "/[0-9]{23}.$name2/";
    $file = fopen("MYFILE.TXT", "r");
    while(!feof($file)){
        $line = fget($file);  

    for ($row = 0; $row = 75; $row++{
    $name2 = data i am getting from another file...;
    $RE21 = "/[0-9]{23}.$name2/";  //Not sure if this works!!
    $a = preg_match($RE21, $line, $matches);
    foreach($matches as $x => $x_value) {
      I will $x_value and store it.} //$x_value should be 00424333060001410100100BILLLLOYD BRRUSSELL 
    }   //foreach  
} //for
}//while
fclose($file);

Why use php for this? You could just `grep` from the command line. slurping in a line, running **75** regexes on it is just... ludicrous. at least with grep you could scan the ENTIRE file for a particular `$name` at once, instead of doing it line-by-painful-line. — Marc B, Jul 15 '15 at 15:19
Marc B...The rest of my code is in PHP...This is only the small part of the entire code. Are you suggesting that I GREP the file for the certain values and dump into a new file (the size could be smaller). And use the new file for the looping. — user2391785, Jul 15 '15 at 15:27

Bobo Wieland · Answer 1 · 2015-07-16T09:02:59.667

1

Maybe you should try a different approach and use command line grep? Generate the regexps from your "another file" and the execute a grep command using your generated pattern and the file you want to search?

Use the -o flag to only get your matches from the result

edited Jul 16 '15 at 09:02

answered Jul 15 '15 at 15:24

Bobo Wieland

70
9

Bobo would it be something like this grep -E "[0-9]{23}[A-Z]\s[A-Z]\s" filename.TXT – user2391785 Jul 15 '15 at 19:42
Now how do I get only what my regex is for? When I do this now I get all other information displayed on the terminal. – user2391785 Jul 15 '15 at 19:43

score 0 · Answer 2 · answered Jul 16 '15 at 09:06

0

You can read it line by line:

$handle = fopen("bla.txt", "r");
while (($buffer = fgets($handle, 4096)) !== false) {
   // ...
}
fclose($handle);

answered Jul 16 '15 at 09:06

radeklos

2,168
21
19

radeklos: The $buffer contains 4096 memory of data each time? I tried this and it takes a while and does timeout. I have a 35mb file with over 650k lines in it. What I am trying to do is, I have file 'A excel', I want to loop thru it get a value and go find it in File B (TXT - 35MB). Once I found it in 'File B' , I want to take it write it in 'FILE A'. I am assuming I put this logic under in your while statement using $buffer variable. Can you explain – user2391785 Jul 19 '15 at 03:24

PHP opening a large file

2 Answers2