How can I use Regex in a While Loop in PHP?

Question

I've to do the Regex in a While loop bc its to much data to do it in once. By now I can go throw the Data and the regex also works but the Data get not stored. So How can I do this?

This is my code now:

$handle = @fopen($PathToFile, "r");
if ($handle) {
    while (($buffer = fgets($handle, 4096)) !== false) {
        if (preg_match_all('/^\w+\s+\d+\s('. $SelectedTime .':\d+.\d+).\d+.\d+\s(.+)/im', $buffer, $matches, PREG_SET_ORDER)) {
            $decodeData[] .= $matches;
        }
        else {

        }
    }

var_dump($decodeData);
}
fclose($handle);

For help I would be really glad

What var_dump return ? I think you should rather use `$decodeData[] = $matches;` instead of `$decodeData[] .= $matches;` — executable, Mar 28 '19 at 14:08
@executable It returns: ```array(301140) { [0]=> string(5) "Array" [1]=> string(5) "Array" [2]=> string(5) "Array" [3]=> ....}``` — , Mar 28 '19 at 14:10
Are you aware that you're potentially closing non opened file? Probably not, you also use `@`... — Marcin Orlowski, Mar 28 '19 at 14:13
u are already storing in array `$decodeData[]` why are you concatenation> `.= ` — devpro, Mar 28 '19 at 14:14
for large file i think u need to store `$buffer` in an array then you can break this array in chunks by using `array_chunks()` then u can use regex and store result in a new array — devpro, Mar 28 '19 at 14:16

score 0 · Answer 1 · edited Mar 28 '19 at 14:13

0

This is wrong syntax, it is not a string concatenation.

Instead of

$decodeData[] .= $matches;

Output:

$decodeData[0] = "Result1Result2Result3Result4";

Use

$decodeData[] = $matches;

Output:

$decodeData[0] = "Result1";
$decodeData[1] = "Result2";
$decodeData[2] = "Result3";
$decodeData[3] = "Result4";

edited Mar 28 '19 at 14:13

Martin

22,212
11
70
132

answered Mar 28 '19 at 14:11

Nik

2,885
2
25
25

If I try this I get: ```PHP Fatal error: Allowed memory size of 268435456 bytes exhausted``` – Mar 28 '19 at 14:13
@User9123 how big is your file? – Martin Mar 28 '19 at 14:14
@Nik - I added output examples to your answer for clarity – Martin Mar 28 '19 at 14:14
How large is your file? – Nik Mar 28 '19 at 14:14
this got nothing with syntax. – Marcin Orlowski Mar 28 '19 at 14:15
@Martin 591973500 bytes, bc its that big I made this loop – Mar 28 '19 at 14:16
for large file i think u need to store $buffer in an array then you can break this array in chunks by using array_chunks() then u can use regex and store result in a new array – devpro Mar 28 '19 at 14:16
@User9123: if you cant then try with `preg_match` instead of `preg_match_all` this will reduce load time. and share the result, – devpro Mar 28 '19 at 14:19
@devpro I get ```preg_match(): Invalid flags specified``` – Mar 28 '19 at 14:22
@User9123 you need to increase `memory_limit` in `php.ini` or using `ini_set` or flush the `$decodeData` array after X iterations somewhere. What is your PHP version? – Nik Mar 28 '19 at 14:23
@Nik I've the version: ```PHP 7.0.33-0ubuntu0.16.04.3 (cli) ( NTS ) Copyright (c) 1997-2017 The PHP Group Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies with Zend OPcache v7.0.33-0ubuntu0.16.04.3, Copyright (c) 1999-2017, by Zend Technologies``` And I cant increase the ```memory_limit``` because its already really high – Mar 28 '19 at 14:25
Then you should think where to store your matches to avoid keep all in memory (php is not good with this). You can write the target matches into a csv file or into a database or into a memcache (depends what is best for you) to use them later. – Nik Mar 28 '19 at 14:31
@Nik I need the matches for some statistic. After the regex I convert the Data to a JSON File. So I've to filter this data. And this works also for smaller files with like 10000 to 30000 Records but if I have like 13000000 Records its crashing – Mar 28 '19 at 15:02
Well, your script simple can not put all the entries into memory (php array). Try to split the files into chunks before processing or use bigger server. Other options I mentioned above. – Nik Mar 28 '19 at 15:09
@Nik please a sample – Mar 28 '19 at 17:44
How to split files see https://stackoverflow.com/questions/2016894/how-to-split-a-large-text-file-into-smaller-files-with-equal-number-of-lines – Nik Mar 28 '19 at 17:50

score 0 · Answer 2 · answered Mar 28 '19 at 14:14

0

You need to use preg_quote function to escape $SelectedTime variable correctly and use $decodedData[] = $matches without .=.

$handle = @fopen($PathToFile, "r");
if ($handle) {
    while (($buffer = fgets($handle, 4096)) !== false) {
        $pattern = '/^\w+\s+\d+\s('. preg_quote($SelectedTime) .':\d+.\d+).\d+.\d+\s(.+)/im';
        if (preg_match_all($pattern, $buffer, $matches, PREG_SET_ORDER)) {
            $decodeData[] = $matches; // just assignment operator
        }
        else {

        }
    }

var_dump($decodeData);
}
fclose($handle);

answered Mar 28 '19 at 14:14

Lucas Campelo

1
2

I get the same Error: ```PHP Fatal error: Allowed memory size of 268435456 bytes exhausted``` – Mar 28 '19 at 14:19
Try to do with a small file, for testing purposes. If works, you can change the `memory_limit` directive of `php.ini` file. – Lucas Campelo Mar 28 '19 at 20:02
Do you now any other way without editing the ```memory_limt``` – Mar 28 '19 at 20:59

How can I use Regex in a While Loop in PHP?

2 Answers2