9

I'm working in c#, and looking for a way to create a path to a directory that will map to an IO.Stream instead of to the actual file system.
I want to be able to "save" files to that path, manipulate the content or file names, and then save them from that path to a regular file in the file system.

I know I can use a temporary file, but I would rather use the memory for both security and performance.

This kind of thing exists, according to this answer, in Java, using the FileSystemProvider class. I'm looking for a way to do it in c#.

I've tried every search I could think of and came up only with the java answer and suggestions to use Temporary files.
Is it even possible using .net?

Basically, I'm looking for a way to enable saving files directly to memory as if they where saved into the file system. so, for instance, if I had a 3rd party class that exposes a save method (save(string fullPath)), or something like the SmtpServer.Send(MyMsg) in this question, i could choose that path and save it into the memory stream instead of onto the drive. (the main thing here is that I want to provide a path that will lead directly to a memory stream).

Community
  • 1
  • 1
Zohar Peled
  • 79,642
  • 10
  • 69
  • 121
  • 8
    How about `Dictionary` mapping file names to memory streams? – Lucas Trzesniewski May 10 '15 at 09:44
  • I probably should have make it more clear, but one of the reasons I want this is so that I could save files to memory from outside sources such as sql backups or framework classes. I don't see how using a Dictionary will enable me to do that. – Zohar Peled May 10 '15 at 09:46
  • Framework classes usually work with `Stream` objects, so you're good with that. As for the outside sources, what would be the way you read the data from them? It may help if you give a *concrete* example. – Lucas Trzesniewski May 10 '15 at 09:48
  • I've actually thought about this as a possible solution to 2 stackoverflow questions: [one on how to save mail messages with a specified name](http://stackoverflow.com/questions/29988040/specify-eml-file-name-using-system-net-mail-mailaddress-or-other-library) and the other one on how to make sure that sql backups was created by the same software (that I can't find right now). This kind of thing might be useful in both cases. – Zohar Peled May 10 '15 at 09:51
  • 1
    If you still don't like using `Dictionary` then you can relatively easy invent your own structure for virtual folders and files that are using `MemoryStream` as a file contents storage. – Dusan May 10 '15 at 09:52
  • @ZoharPeled If you are saving multiple (large) files such as sql backups to memory, you will be (in a way) using temporary files anyway, because the system will likely write much of the virtual memory to disk. You may want to use temporary [memory mapped files](https://msdn.microsoft.com/en-us/library/system.io.memorymappedfiles.memorymappedfile%28v=vs.110%29.aspx) to make this process more efficient. – John Colanduoni May 10 '15 at 09:52
  • @ZoharPeled, with your comment about two other stackoverflow questions you have started to reveal the real problem. What is the exact, real problem you have you are trying to solve? – Dusan May 10 '15 at 09:57
  • @Dusan Thanks, But the problem is how do i give this structure a path I can use to specify backup or save locations? – Zohar Peled May 10 '15 at 09:58
  • @JohnColanduoni I know, but this is something internal to the OS, I doubt if I can change that in .Net. – Zohar Peled May 10 '15 at 09:59
  • @Dusan: It's more of a curiosity question. I just thought that these 2 questions might be solved with a temporary, memory based path to save the file into, thus allowing the software to manipulate the file structure or name before actually saving it into the file system. – Zohar Peled May 10 '15 at 10:02
  • `class VirtualFile { string SrcFileName; string DestFileName; MemoryStream Contents; ... }` `class VirtualDirectory { string DirName; List Files; ... }` Something like that, invent your own structure per your needs, shape it as you wish. Couple of classes, very easy. – Dusan May 10 '15 at 10:05
  • @Dusan Thanks but, I've looked at [`VirtualFile`](https://msdn.microsoft.com/en-us/library/system.web.hosting.virtualfile%28v=vs.110%29.aspx) and it's friends and I'm not sure it's what I was going for. will edit my question so that it will be more clear what I'm asking for. – Zohar Peled May 10 '15 at 10:25
  • But Word is not set up to read or write to a memory stream. You can't just change the rules of the Windows file system. – paparazzo May 10 '15 at 11:02
  • Maybe word is not the best example. I'll try to change to a better example – Zohar Peled May 10 '15 at 11:23
  • @ZoharPeled Have you looked at solutions such as redis? Seems to me you can storage the relevant "path" as a key, and serialize the data to binary and store it as a value. – Yuval Itzchakov May 10 '15 at 11:42
  • @YuvalItzchakov Thanks, but I don't see how that helps. the problem is not storing the data, the problem is giving the stimulated `path` as a way to save data directly into a stream as if it was a file. – Zohar Peled May 10 '15 at 11:47
  • *the problem is giving the stimulated path as a way to save data directly into a stream as if it was a file*. I don't understand what your problem is then. You can take a path of a given key and store that data inside a stream, instead of a file. – Yuval Itzchakov May 10 '15 at 11:49
  • but how do I expose that path as if it was a regular file path in the file system? suppose you want to manipulate an sql server .bak file. you can use backup command in sql server, then open the file, change whatever you want and save the file back to the file system. what I'm looking for is a way to achieve the same result but without having to save the file to the file system first. I hope it's clear now. I want to have a path like "z:/Myfolder" that instead of pointing to a place on a disk it will point to a memory stream. – Zohar Peled May 10 '15 at 11:53
  • If Word is a bad example then how is SQL backup command a good example? Just point z:/Myfolder to a memory stream does not mean these non .NET apps are gong to magically know how to write to a .NET memory stream. – paparazzo May 10 '15 at 13:26
  • @Blam: Find me a way to do the pointing, I'll find a way to translate the stream. after all, it's all ones and zeros at the end of the day... – Zohar Peled May 10 '15 at 13:28
  • So what if it is ones and zeros. How are you going to get Word to write to .NET memory stream? It is not the same API. – paparazzo May 10 '15 at 16:56
  • I admit I didn't think about that when I asked the question, but all files are written to the file system are either binary or text files, right? right now let's assume I'm only interested in text files. surly there must be a simple way to use text written into a file as if it was written into a stream, right? if it's possible in java i would expect it should be possible in c# as well... – Zohar Peled May 10 '15 at 17:00
  • What do you mean it is possible in java? Writing to memory from a given application is not the same as an arbitrary application writing to virtual directory .NET memory stream. – paparazzo May 11 '15 at 16:53
  • My question is the same is the post I've linked to, the only difference is that it's in c# and not java. – Zohar Peled May 11 '15 at 18:18
  • Perhaps I didn't explain myself very well, I was looking for a way to save files into memory instead of the file system, so that I can manipulate these files before actually saving them into the file system. It doesn't have to be a MemoryStream, I just figured that if there is a way to do it, then it will involve a MemorySteam. Anyway, I'm kinda over that, as I understand it's not as easy in c# as it looks like in Java, according to that post. – Zohar Peled May 13 '15 at 06:21

2 Answers2

9

.NET doesn't have an abstraction layer over the host OS's file system. So unless you can build your own for use in custom code, and you need to have 3rd party libraries covered, there are just two workable optilns:

  • Use streams and avoid any APIs working with file names.
  • Build a virtual file system plugged into your host OS's storage architecture; however, the effort needed versus benefits is highly questionable.
Ondrej Tucny
  • 27,626
  • 6
  • 70
  • 90
  • 1
    So, the answer is that it can't be done, then? I'll wait a little while to see if anyone can come up with a solution before accepting your answer, if you don't mind. – Zohar Peled May 10 '15 at 15:35
  • No, it definitely can be done, but it is a huge amount of work and requires intimate knowledge of how file systems and the windows driver system works. – Lasse V. Karlsen May 10 '15 at 16:58
  • 1
    Ok. In that case, It, seems like I've tried to byte more than I can swallow. Thanks to everyone that wrote on this question. – Zohar Peled May 10 '15 at 18:19
1

I went through a similar situation lately, and there is no out of the box solution in .NET for doing that although I used a workaround which was efficient and safe for me.

Using Ionic.Zip Nuget package you can create a whole directory with a complex structure as a stream in memory and although it will be created as a zip file, you can extract it as a stream or even send the zip file as a stream.

    using (var zip = new Ionic.Zip.ZipFile())
                {
                    zip.AddEntry($"file1.json", new MemoryStream(Encoding.UTF8.GetBytes(someJsonContent)));

                        for (int i = 0; i < 4; i++)
                        {
                            zip.AddEntry($"{myDir}/{i}.json", new MemoryStream(Encoding.UTF8.GetBytes(anotherJsonContent)));
                        }
                }

And here is how to extract a zip file as a stream using Ionic.Zip

Marzouk
  • 2,650
  • 3
  • 25
  • 56