2

I am using PHP 7.1 (if it makes any difference), and I am looking for possible issues with opening a file handler with fopen, but closing that file handler a long time after (potentially 30+ minutes).

The scenario comes down to a long running script that logs its actions periodically. Right now, I use a simple process of using file_put_contents with FILE_APPEND. It works. But it slows down over time as the file gets bigger and bigger.

Using fopen and fwrite means I can avoid that slow-down. The only issue is that I wouldnt be calling fclose until the end of the scripts execution. I assume constantly fopen/fclose'ing the file will give me the same poor performance as my current implementation.

Any one have experience with the internals in this regard? I have no need to read the file, only write.

EDIT: More info: I am currently running this in a Linux VM on my laptop (VirtualBox). I do not expect top notch performance from this setup. That being said, I still notice the slow-down as the log file gets bigger and bigger.

My code logic as as simple as:

while(true)
{
    $result = someFunction();
    if(!$result)
    {
        logSomething();
    }
    else
    {
        break;
    }
}

The frequency of writes is several times a second.

And files can become several GBs in size.

Kovo
  • 1,687
  • 14
  • 19
  • Did you write some code and ran into memory issues? – Ali Gajani Oct 19 '17 at 18:21
  • https://stackoverflow.com/questions/17391747/if-php-script-reaches-max-allowed-exec-time-is-it-terminated-immediately-or-rig could be related and https://stackoverflow.com/questions/14423096/error-writing-to-a-text-file-using-fwrite-and-file-put-contents – Funk Forty Niner Oct 19 '17 at 18:22
  • No memory issues. Just a slow-down in write performance. – Kovo Oct 19 '17 at 18:22
  • It could be related to either the code you're using, or your system. Could you post more detail and possibly the code you're using? as well as the file's size when it starts losing performance. – Funk Forty Niner Oct 19 '17 at 18:25
  • How big is the file? I've written huge log files one line at a time with `file_put_contents` without any issues... And have you considered simply rotating the log sometimes? Your comment that it logs "periodically" suggests that it is not written to that often, is that correct? – Mikk3lRo Oct 19 '17 at 18:26
  • why do you think it is php-related ? could be your filesystem or OS as well. – Calimero Oct 19 '17 at 18:29
  • I just updated my question with some more environment info, as well as a basic example of what I am doing. As for frequency, it writes several times a second. – Kovo Oct 19 '17 at 18:30
  • I have not ruled-out a system related issue. Which is why I am posing the question. If there should be no perceived slow-down, then perhaps it is a system problem. – Kovo Oct 19 '17 at 18:31
  • to assert this you might want to try a simple, equivalent script in another language, just to narrow down the perimeter. Just a big loop and file writes. Might be good to do it in php too to get a MCVE. – Calimero Oct 19 '17 at 18:35
  • Have you tried isolating the problem. You might fx. make a script that does **nothing** other than logging something at a similar pace and check if that exibits the same symptoms after running for a while / when reaching the same filesize. You can then use the same script to confirm that it goes away if you switch to `fopen`. I don't think it should be a problem in itself to keep a file open for a long time. – Mikk3lRo Oct 19 '17 at 18:37

0 Answers0