7

How can i get a particular line in a 3 gig text file. The lines are delimited by \n. And i need to be able to get any line on demand.

How can this be done? Only one line need be returned. And i would not like to use any system calls.

Note: There is the same question elsewhere regarding how to do this in bash. I would like to compare it with the PHP equiv.

Update: Each line is the same length the whole way thru.

JavaRocky
  • 19,203
  • 31
  • 89
  • 110
  • After updating, i was thinking, maybe using some sort of way to precalculate and fetch exactly which characters in the file to get back. – JavaRocky May 08 '10 at 12:44
  • You want a database, without the database. Are you sure you can't just use a database? – o0'. May 08 '10 at 12:55
  • You want to read a file without *any system calls*... sorry but that just is not possible. (Did you mean without using `exec` and the like?) – salathe May 08 '10 at 17:21
  • Ah, the update really helps....! – zaf May 08 '10 at 17:45
  • possible duplicate of [How to save memory when reading a file in Php ?](http://stackoverflow.com/questions/2603807/how-to-save-memory-when-reading-a-file-in-php) – Gordon May 08 '10 at 17:51
  • May I ask what that file is? 3gb?? – ggfan May 08 '10 at 18:14

5 Answers5

8

Without keeping some sort of index to the file, you would need to read all of it until you've encountered x number of \n characters. I see that nickf has just posted some way of doing that, so I won't repeat it.

To do this repeatedly in an efficient manner, you will need to build an index. Store some known file positions for certain (or all) line numbers once, which you can then use to seek to the right location using fseek.

Edit: if each line is the same length, you do not need the index.

$myfile = fopen($fileName, "r");
fseek($myfile, $lineLength * $lineNumber);
$line = fgets($myfile);
fclose($myfile);

Line number is 0 based in this example, so you may need to subtract one first. The line length includes the \n character.

Thorarin
  • 47,289
  • 11
  • 75
  • 111
8

There is little discussion of the problem and no mention is made of how the 'one line' should be referenced (by number, some value within it, etc.) so below is just a guess as to what you're wanting.

If you're not averse to using an object (it might be 'too high level', perhaps) and wish to reference the line by offset, then SplFileObject (available as of PHP 5.1.0) could be used. See the following basic example:

$file = new SplFileObject('myreallyhugefile.dat');
$file->seek(12345689); // seek to line 123456790
echo $file->current(); // or simply, echo $file

That particular method (seek) requires scanning through the file line-by-line. However, if as you say all the lines are the same length then you can instead use fseek to get where you want to go much, much faster.

$line_length = 1024; // each line is 1 KB line
$file->fseek($line_length * 1234567); // seek lots of bytes
echo $file->current(); // echo line 1234568
salathe
  • 51,324
  • 12
  • 104
  • 132
1

The only way I can think to do it would be like this:

function getLine($fileName, $num) {
    $fh = fopen($fileName, 'r');

    for ($i = 0; $i < $num && ($line = fgets($fh)); ++$i);

    return $line;
}
Rup
  • 33,765
  • 9
  • 83
  • 112
nickf
  • 537,072
  • 198
  • 649
  • 721
1

You said each line has the same length, so you can use fopen() in combination with fseek() to get a line quickly.

http://ch2.php.net/manual/en/function.fseek.php

svens
  • 11,438
  • 6
  • 36
  • 55
0

While this is not a solution exactly, how come you are needing to pull out one line from a 3 gig text file? is perfomance an issue or can this run a leisurely pace? If you need pull lots of lines out of this file at different points in time, i would definately suggest putting this data into a DB of some kind. SQLite maybe your friend here as its very simple but not great with lots of scripts/people accessing it at one time.

James Butler
  • 3,852
  • 1
  • 26
  • 38