-2

The text file (from which my function grab info) is being recreated everyday at 00:00 and my application starts slowly during the day to add lines by lines as log file. The file is approximately 1 mb and with around 6,000 lines at the end of the day. So, it's not really small file. The function is grabbing latest 21 lines from the text file to display latest 21 death/kills for a game and the script can be accessed by a lot of users to check the log statistics. My question is how I can make the script with better performance because file() is reading the whole file actually lines by lines, so what's a better way to do this? Can it affect the performance if a lot of people access the log page to display this info from the text file or for 6,000 lines, it should be all good ? The file is as plain text in .txt format and these are an example lines from the file:

1,42,16, 201,stackoverflow_user, 1, 6762160, 39799, 9817242, 6762160, 39884, 10010545,stackoverflow_user, 2, 1351147, 1165, 483259, 1351147, 1115, 241630, 0 
1,46,27, 201,[stackoverflow_user | stackoverflow_userother], 1, 4078465, 286991, 1594830, 4078465, 287036, 1643156,stackoverflow_user, 2, 1357147, 1115, 241630, 1357147, 1065, 120815, 0

My function:

# read a file into an array
$lines = file('C:/path/to/file.txt', FILE_IGNORE_NEW_LINES | FILE_SKIP_EMPTY_LINES);

# flip our array over so the last lines of the file are first.
$lines = array_reverse($lines);
$n = 1;
$wanted = 21; # or however many lines you want.
$content = '';

foreach ($lines as $l) {
   # treat the data as comma-separated values
   $arr = explode(",", $l);
   # if col 5 has multiple values, take the first one
   if (preg_match("/\[(.+?) \|/", $arr[4], $matches)) {
       $arr[4] = $matches[1];
   }
   # is arr[4] the same as arr[12]?
   if ($arr[4] !== $arr[12]) {
       # these two are not equal, so use these values
       $data = array('rank-pos' => $n++, 'rank-name' => $arr[4], 'rank-dmuser' => $arr[12]);
       $content .= Template::Load('rankinguserdm-' . ($n % 2 == 1 ? 2 : 1), $data);
   }
   # have we got enough data?
   if ($n === $wanted) {
       break;
   }
}

        $this->content = Template::Load('user_rankingsdm', array('rankings' => $content));
    }
Monk25
  • 25
  • 1
  • 6
  • 5
    Why don't you use a database? SQLite comes immediately to mind … – nietonfir Oct 17 '14 at 20:11
  • I can't use database becuase the application is writing info to that text file and I must use the function to grab that info and display it on my website. Also, I am running on MSSQL Server 2005. So, as my question, is `file()` fine for that purpose (just my concern is if more people access the log statistics page to view the logs and hence overload the server?!) or maybe I must use `fopen` or something ? – Monk25 Oct 17 '14 at 20:12
  • You might be interested in reading http://stackoverflow.com/questions/16741258/how-can-i-read-only-parts-of-text-file – SolarBear Oct 17 '14 at 20:15
  • Yeah, I get that. But you could write a simple service (run by a daemon e.g. all 30s) that reads the file and stores it in a database. And then your application reads from there with all its benefits. – nietonfir Oct 17 '14 at 20:16
  • So, it must reads it every second which will overload the database in that case ... at least affect it's performance ... – Monk25 Oct 17 '14 at 20:17
  • 1
    I'm pretty sure that a database can handle your requirements quite well and will not "overload". And they're a lot more scaleable than that file solution. ;-) – nietonfir Oct 17 '14 at 20:27
  • So, basically you say instead of reading the file and display ... it must read it every second and add data to the table ? I am using also MSSQL Server 2005 – Monk25 Oct 17 '14 at 20:29
  • Using a database is *way* better option. Trust us, it can handle the load just fine. However, this question [nerd sniped](http://xkcd.com/356/) me. https://gist.github.com/Sammitch/7ff9c8486f8317666f4d – Sammitch Oct 17 '14 at 21:17
  • Okay, thanks. I understand. But simply, my question is even if I use the above function, will it realy affect the performance for small files like 1mb and around 6,000 lines ? Please let me know. – Monk25 Oct 17 '14 at 21:39
  • Yes, because you're reading in 5,979 useless lines as opposed to only the 21 you need. – Sammitch Oct 17 '14 at 22:18
  • So, will you be able to help me out tweak my current function to read only the latest lines? I will be so grateful to help me out. Thanks for your time and effort! – Monk25 Oct 17 '14 at 22:27

1 Answers1

-1

if you are running on a unix system, think of calling tail

<?php
$file = log.txt;
$last_lines = `tail -n 21 $file`; // backticks indicate system call
print_r($last_lines);
PaulH
  • 2,918
  • 2
  • 15
  • 31
  • I am running on Windows Server 2008 R2 Enterprise. So, `tail` isn't option for me. :/ – Monk25 Oct 17 '14 at 20:15
  • Can you use a windows equivalent? http://stackoverflow.com/questions/187587/looking-for-a-windows-equivalent-of-the-unix-tail-command – PaulH Oct 17 '14 at 20:22
  • I understand but will this have a better performance boost than `file()` when my files are approximately 1 mb in size with 5,000 - 6,000 lines? – Monk25 Oct 17 '14 at 20:26