Read and write to the same csv file

Question

I have a CSV file (E.g. Directories.csv) which contains a huge list of directories. I am looping through the directories from the CSV using streamreader and performing some task. I am updating the completed directory list to a dictionary and stuck at this step now.

Ask: I want to capture the data through the loop on which directories are complete in the same CSV just in case the application crashes or server reboots, so that I don't have to re-iterate through the loop again which got completed. (Or) Delete the completed directories row from the CSV

I tried to check online for suggestions and asking to create temp file and move the copy of it. Can this be possible in case the server reboots or application crashes? Please suggest how can I take this forward.

My code:

  Dictionary<string, string> directoryDictionary = new Dictionary<string, string>();
  using (FileStream fileStreamDirectory = File.Open(outputdir + "\\Directories.csv", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
  using (BufferedStream bufferStreamDirectory = new BufferedStream(fileStreamDirectory))
  using (StreamReader streamReaderDirectory = new StreamReader(bufferStreamDirectory))
  {
      while ((Directoryline = streamReaderDirectory.ReadLine()) != null)
      {
        #Doing the task here
        directoryDictionary.Add(Directoryline, "Completed");
      }
  }

Answer 1

You can't really insert data into middle of a text file (unless it is fixed width format which in not the case of CSV).

Two options:

• read to memory, update in-memory data, rewrite whole table back to the file (may need to keep previous version in case of write failures)
• use database that satisfy you criteria and import CSV there to work with.