Read text file at specific line

Question

I have a text file which has more than 3000 lines. I am finding the number of lines using

string[] lines = File.ReadAllLines(myPath);
var lineCount = lines.Length; 

Then I am generating a random number

Random rand = new Random();
var lineToRead = rand.Next(1, lineCount);

Now I need to read the specific line that is generated by random number. I can do this using

string requiredLine = lines[lineToRead];

Because my file is big I don't think creating such a big array is efficient. Is there a more efficient or easier way to do this?

Answer 1

Here is a solution which iterates the file twice (first time to count lines, next time to select line). The benefit is that you don't need to create an array of 3000 strings in memory. But, as mentioned above, it will possibly be slower. Why possibly? - because File.ReadAllLines creates a list of strings inside and that list will be resized many times while filling it with 3000 items. (Initial capacity will be 4. When the inner array is completely filled, then the new array of doubled size will be created and all strings will be copied there).

So, the solution uses File.ReadLines method which returns IEnumerable<string> with lines and skip lines you don't need:

IEnumerable<string> lines = File.ReadLines(myPath);
var lineToRead = rand.Next(1, lines.Count());
var line = lines.Skip(lineToRead - 1).First();

BTW, internally File.ReadLines uses SteamReader which reads file line by line.

Answer 2

What you can do is parse the file to find the index of each line and then at a later time you can go back to a certain line by using Stream.Position to get the content. Using this method you don't need to keep anything in memory and it is reasonably fast. I tested this on a file that is 20K lines and 1MB in size. It took 7ms to index the file and 0.3to get the line.

    // Parse the file
    var indexes = new List<long>();
    using (var fs = File.OpenRead("text.txt"))
    {
        indexes.Add(fs.Position);
        int chr;
        while ((chr = fs.ReadByte()) != -1)
        {
            if (chr == '\n')
            {                        
                indexes.Add(fs.Position);
            }
        }
    }

    int lineCount = indexes.Count;
    int randLineNum = new Random().Next(0, lineCount - 1);
    string lineContent = "";


    // Read the random line
    using (var fs = File.OpenRead("text.txt"))
    {
        fs.Position = indexes[randLineNum];
        using (var sr = new StreamReader(fs))
        {
            lineContent = sr.ReadLine();
        }
    }

Answer 3

You can wrap your stream into StreamReader and call ReadLine as many times as needed to go to your target line. That way you don't need to hold the whole file contents in memory.

However, this is only feasible if you do that rarely and file is quite big.

Answer 4

Use Reservoir Sampling to solve this in in a single pass

If you want to randomly choose one or more items from a list of items where the length of that list is not known in advance, you can use Reservoir Sampling.

We can take advantage of that, along with the File.ReadLines() method (which avoids buffering all the lines in memory) to write a single-pass algorithm that will read each line just once, without buffering.

The sample code below shows a generalised solution that lets you randomly select any number of lines. For your case, N = 1.

The sample code also includes a test program to prove that the lines are chosen randomly with a uniform distribution.

(To see how this code works, see the Wiki article I linked above.)

using System;
using System.IO;
using System.Collections.Generic;

namespace Demo
{
    internal class Program
    {
        public static List<string> RandomlyChooseLinesFromFile(string filename, int n, Random rng)
        {
            var result = new List<string>(n);
            int index = 0;

            foreach (var line in File.ReadLines(filename))
            {
                if (index < n)
                {
                    result.Add(line);
                }
                else
                {
                    int r = rng.Next(0, index + 1);

                    if (r < n)
                        result[r] = line;
                }

                ++index;
            }

            return result;
        }

        // Test RandomlyChooseLinesFromFile()

        private static void Main(string[] args)
        {
            Directory.CreateDirectory("C:\\TEST");
            string testfile = "C:\\TEST\\TESTFILE.TXT";
            File.WriteAllText(testfile, "0\n1\n2\n3\n4\n5\n6\n7\n8\n9");
            var rng = new Random();
            int trials = 100000;
            var counts = new int[10];

            for (int i = 0; i < trials; ++i)
            {
                string line = RandomlyChooseLinesFromFile(testfile, 1, rng)[0];
                int index = int.Parse(line);
                ++counts[index];
            }

            // If this algorithm is correct, each line should be chosen
            // approximately 10% of the times.

            Console.WriteLine("% times each line was chosen:\n");

            for (int i = 0; i < 10; ++i)
            {
                Console.WriteLine("{0} = {1}%", i, 100*counts[i]/(double)trials);
            }
        }
    }
}

Answer 5

Below will help you in reading at specific line in a file..

http://social.msdn.microsoft.com/Forums/en-US/csharpgeneral/thread/4dbd68f6-61f5-4d36-bfa0-5c909101874b

A code snipet

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;

namespace ReadLine
{
class Program
{
    static void Main(string[] args)
    {
        //Load our text file
        TextReader tr = new StreamReader("\\test.txt");

        //How many lines should be loaded?
        int NumberOfLines = 15;

        //Make our array for each line
        string[] ListLines = new string[NumberOfLines];

        //Read the number of lines and put them in the array
        for (int i = 1; i < NumberOfLines; i++)
        {
            ListLines[i] = tr.ReadLine();
        }

        //This will write the 5th line into the console
        Console.WriteLine(ListLines[5]);
        //This will write the 1st line into the console
        Console.WriteLine(ListLines[1]);

        Console.ReadLine();

        // close the stream
        tr.Close();
    }
}
}

These can also be helpful..

http://www.tek-tips.com/viewthread.cfm?qid=1460456

How do I read a specified line in a text file?

And below is for editing

Edit a specific Line of a Text File in C#

Hope it helps...

Answer 6

you can try like below... it can not create any big array but get a particular line...

string path = "D:\\Software.txt";
int lines = File.ReadAllLines(path).Length;
Random rand = new Random();
var lineToRead = rand.Next(1, lines);
var requiredLine = System.IO.File.ReadLines(path).Skip(lineToRead - 1).First();
Console.WriteLine(requiredLine.ToString());