Is there a memory efficient way to use 'using' within a recursive function when e.g. writing lines to a file?

Question

Is there a memory efficient way to use 'using' within a recursive function when e.g. writing lines to a file?

I read C# 'using' block inside a loop and it mentioned that you don't want to put a using statement inside a for loop unless you have to. (makes sense, one doesn't want multiple instances of 'using' if one doesn't need them). So in the case of a for loop if you can put it outside, you do.

But here, I have a recursive function. So the 'using' statement is going to run multiple times even if I put it outside of a for.

So what is a good or proper way of placing the 'using' statement?

I don't know if I should avoid 'using', and declare the StreamWriter object, StreamWriter writetext before the method call and dispose of it after with writetext.Dispose(). Or maybe there is a more conventional way with 'using'. Maybe wrapping the 'main' call DirSearch_basic_writetofile("c:\\aaa"); with a 'try' and putting the Dispose line in a finally. And avoiding 'using' then. That's just a thought.

// requires directory c:\texts
File.Delete(@"c:\texts\filelist.txt");
// list files and subdirectories of c:\aaa and write them to file "c:\texts\filelist.txt"
DirSearch_basic_writetofile("c:\\aaa");

// recursive function that lists files  and directories and subdirectories,in given directory  

static void DirSearch_basic_writetofile(string sDir)
{
    Console.WriteLine("DirSearch(" + sDir + ")");
    Console.WriteLine(sDir+@"\");
    try
    {
        using (StreamWriter writetext = new StreamWriter("c:\\texts\\filelist.txt",true))
        {
            writetext.WriteLine("DirSearch(" + sDir + ")");
            writetext.WriteLine(sDir);

            foreach (string f in Directory.GetFiles(sDir))
            {
                Console.WriteLine(f);
                writetext.WriteLine(f);
            }
        }

        foreach (string d in Directory.GetDirectories(sDir))
        {
            DirSearch_basic_writetofile(d);
        }

    }
    catch (System.Exception excpt)
    {
        Console.WriteLine(excpt.Message);
    }

}

Answer 1

The linked thing is a case where you were using the same resource for all itterations of the loop. In that case, opening and closing it every itteration serves no purpose. As long as it is closed by the end of all loops, it is save enough.

The opposite case is if you use a different resource every itteration. Say, when going over a list of filenames or full paths, to open each in turn. In that case you got no chocie but to have a new File related instance each itteration.

A recursion is not really different from a loop. You can always replace a loop with a recursion, but the opposite is not always the case. Same rules apply:

• If it is the same resource, you just have to move the creation of the resource outside the recursive function. Rather then taking a path (or using a hardcoded one), let it take a Stream. That keeps the function nicely generic
• If you got a different resource, you have no choice but create a new Instance with a new using every recursion. However I can not think of any "recursive using" case.

If you got to itterate over all files in a directory inlcuding all subdirectories, you would have the recursive function recurse over the directories (no unmanaged resource needed). And then a loop inside the recursive function to itterate over the files inside the current directory (wich requires unmanaged resources).

Edit:

static void DirSearch_basic_writetofile(string currentDirectory, StreamWriter Output){
    //do your thing, using output as the stream you write to

    //Do recusirve calls as normal
    DirSearch_basic_writetofile(subDir, Output);
}

calling it:

using (StreamWriter OutputWriter = new StreamWriter("c:\\texts\\filelist.txt",true){
    DirSearch_basic_writetofile(startDirectory, OutputWriter);
}

Answer 2

If we want to solve using yield return

You might want to restructure the code such that you separate out the recursive part; for example with a yield return.

Something like this below ( sorry, no IDE at hand, let's see if this works) is a simplistic approach.

If you need to write out the new header ( DirSearch(" + sDir + ") ) every time you switch directory, that's doable by not returning String only from producer an object containing String directoryName, List fileNames, and return only once for each directory.

static void DirSearch_basic_writetofile(string sDir)
{
    Console.WriteLine("DirSearch(" + sDir + ")");
    Console.WriteLine(sDir+@"\");
    IEnumerable<String> producer = DirSearch_Producer(string sDir);
    try
    {
        using (StreamWriter writetext = new StreamWriter("c:\\texts\\filelist.txt",true))
        {
            writetext.WriteLine("DirSearch(" + sDir + ")");
            writetext.WriteLine(sDir);

            foreach (string f in DirSearch_Producer(sDir))
            {
                Console.WriteLine(f);
            }
        }
    }
    catch (System.Exception excpt)
    {
        Console.WriteLine(excpt.Message);
    }
}

public static IEnumerable<String> DirSearch_Producer(string sDir){
  foreach (string f in Directory.GetFiles(sDir))
  {
    yield return f;
  }
  foreach (string d in Directory.GetDirectories(sDir))
  {
    foreach (String f in DirSearch_Producer(d)){
        yield return f;
    }
  }
}

Alternative, without using yield return we can use the Directory.GetFiles with EnumerationOptions to go through subdirectories as well. It makes things much simpler. See: RecurseSubdirectories

Answer 3

I am going to leave Christopher's answer as accepted..

I will just note here the solution I used..

using (writetext = new StreamWriter("c:\\texts\\filelist.txt", true))
   DirSearch_basic_writetofile_quicker("c:\\aaa");

I also used a StringBuilder and wrote to console and file after

        // https://stackoverflow.com/questions/15443739/what-is-the-simplest-way-to-write-the-contents-of-a-stringbuilder-to-a-text-file
        System.IO.File.WriteAllText(@"c:\texts\filelist.txt", sb.ToString());

        Console.Write(sb.ToString());

mjwillis had suggested to me that writing to the console one line at a time was a bottleneck, and christopher mentioned about writing to the file at the end. So I just write to both at the end / after the call.

I used a static variable for StreamWriter and a static variable for StringBuilder, so that main() and the recursive function could see it. I didn't want to create a new parameter for the recursive function, 'cos I wanted to keep the recursive call looking 'simple' (one parameter).