Possible to create a file in memory?

Question

What I need to do is create a file in memory, write to that file, and then access that file as you would with any other file.

Here I am creating and using an actual file:

if (!File.Exists(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")))
{
    File.Create(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")).Dispose();
    using (TextWriter tw = new StreamWriter(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")))
    {
        foreach (string[] array in listOfRows)
        {
            string dataString = array[0] + "," + array[1] + "," + array[2] + "," + array[3];
            tw.WriteLine(dataString);
        }
        tw.Close();
    }
}
else if (File.Exists(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")))
{
    using (TextWriter tw = new StreamWriter(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")))
    {
        foreach (string[] array in listOfRows)
        {
            string dataString = array[0] + "," + array[1] + "," + array[2] + "," + array[3];
            tw.WriteLine(dataString);
        }
        tw.Close();
    }
}

//makes virtual table to be copied to database
DataTable dt = new DataTable();
string line = null;
int i = 0;
using (StreamReader sr = File.OpenText(Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Desktop), "TEST.csv")))
{
    //etc etc etc
}

So as you can see, currently I am just creating a file and then writing data to it, and then using that file. It would be much better for me if I could just do all this in virtual memory, is this possible?

Why do you feel it would be much better? Unless a specific performance issue is present, you may be trying to fix a problem that doesn't exist. — tnw, Aug 19 '14 at 16:32
Please take some time to fix up the indentation of your code - it's all over the place at the moment. — Jon Skeet, Aug 19 '14 at 16:34
"do all this in virtual memory" - as in "I want to use array instead of an instance of strongly typed class, write it to CSV without proper encoding and than re-read whole file to `DataTable` instead of adding single line to it"? There are better ways for each part... and even than beating files with custom in-memory code requires more effort that most people are willing to put in in. — Alexei Levenkov, Aug 19 '14 at 17:10

score 5 · Answer 1 · answered Aug 19 '14 at 16:34

5

Well, you could read/write to a byte array using a MemoryStream. You don't get "lines" with this approach, but its close.

That being said, it sounds like you are way down the rabbit hole if you are writing and reading a file right away. Files are for persistence.

If you just want to store a collection of lines of strings in memory, you want to use a StringBuilder or perhaps just a List<T>.

answered Aug 19 '14 at 16:34

BradleyDotNET

60,462
10
96
117

+1 Despite you giving the other answer a +1 for "giving him what he wants, not what he probably needs", and dispite me often being irritated when I get an answer or comment on one of my questions that questions whether I'm sane to be asking for what I'm asking, I think it is very good in this case that you point out that it's unlikely that what he's asking for is what he really needs. – RenniePet Aug 19 '14 at 17:12
+1. Note that `MemoryStream` is fine for small amount of data - larger data sets will hit performance issues with the way `MemoryStream` manages memory (copy on grow instead of chunked storage). – Alexei Levenkov Aug 19 '14 at 17:12

score 4 · Answer 2 · answered Aug 19 '14 at 16:47

Sounds like you want a memory mapped file (.Net 4 or higher needed) From MSDN:

using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;

class Program
{
    static void Main(string[] args)
    {
        long offset = 0x10000000; // 256 megabytes 
        long length = 0x20000000; // 512 megabytes 

        // Create the memory-mapped file. 
        using (var mmf = MemoryMappedFile.CreateFromFile(@"c:\ExtremelyLargeImage.data", FileMode.Open,"ImgA"))
        {
            // Create a random access view, from the 256th megabyte (the offset) 
            // to the 768th megabyte (the offset plus length). 
            using (var accessor = mmf.CreateViewAccessor(offset, length))
            {
                int colorSize = Marshal.SizeOf(typeof(MyColor));
                MyColor color;

                // Make changes to the view. 
                for (long i = 0; i < length; i += colorSize)
                {
                    accessor.Read(i, out color);
                    color.Brighten(10);
                    accessor.Write(i, ref color);
                }
            }
        }
    }
}

public struct MyColor
{
    public short Red;
    public short Green;
    public short Blue;
    public short Alpha;

    // Make the view brighter. 
    public void Brighten(short value)
    {
        Red = (short)Math.Min(short.MaxValue, (int)Red + value);
        Green = (short)Math.Min(short.MaxValue, (int)Green + value);
        Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
        Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
    }
}

+1 for doing exactly what he asked. – BradleyDotNET Aug 19 '14 at 17:01 — BradleyDotNET, Aug 19 '14 at 17:01

Possible to create a file in memory?

2 Answers2