24

I'm writing little program in plain C here. What I need is to create a file directly inside memory (not written on hard disk) Currently I can use fopen("filename.txt,"wb") to write to the file.

I know that in linux you can use fmemopen(). Is there a similar solution for win32?

Graham Borland
  • 60,055
  • 21
  • 138
  • 179
J Nash
  • 341
  • 1
  • 2
  • 4

4 Answers4

21

fdopen is in POSIX.1-1990, and Windows supports it in the form of _fdopen. Using it along with _get_osfhandle and _open_osfhandle, it allows to build up a proper FILE * from a Windows file HANDLE that works with the rest of the C standard library. See also the question Is there a Windows equivalent to fdopen for HANDLEs? for more information regarding this.

Now, the remaining part of the problem is creating a memory-backed Windows file. It turns out there are different approaches to this which approximate this under NT. The most common one seems to be:

  1. Creating a temporary file; i.e. non-persistent. In other words, use CreateFile with a path built from GetTempPath and possibly GetTempFileName.

  2. ...with the FILE_ATTRIBUTE_TEMPORARY file attribute set [1]; i.e. memory-backed as much as possible.

  3. ...and with the FILE_FLAG_DELETE_ON_CLOSE flag set [2]; i.e. hinting that it will never need to be written unless it does not fit in memory.

As long as there is enough memory to keep the file in memory (i.e. low enough memory pressure), the disk will not be used. See this MSDN blogpost from Larry Osterman which describes this approach. Amusingly, he calls such files "temporary" temporary files.

Note that this is not exactly equivalent to fmemopen, which will fail if the file does not fit into the predefined memory area [3] (i.e. fmemopen sets up a pure memory-backed file). However, for many purposes it is close enough (and sometimes the potential backing on disk may be wanted).

In addition, see How to prevent flushing to disk of a memory map opened on a windows temporary delete-on-close file discusses for some discussion regarding these file attributes and other possible approaches.

Finally, for a full example that combines both parts (i.e. creating the memory-backed file and then wrapping it up in a FILE * handle), take a look at the implementations from the fmem library (which was written to provide a cross-platform solution for this problem) or at the one from the libconfuse library.


[1]

Specifying the FILE_ATTRIBUTE_TEMPORARY attribute causes file systems to avoid writing data back to mass storage if sufficient cache memory is available, because an application deletes a temporary file after a handle is closed. In that case, the system can entirely avoid writing the data. Although it does not directly control data caching in the same way as the previously mentioned flags, the FILE_ATTRIBUTE_TEMPORARY attribute does tell the system to hold as much as possible in the system cache without writing and therefore may be of concern for certain applications.

[2]

The file is to be deleted immediately after all of its handles are closed, which includes the specified handle and any other open or duplicated handles.

[3]

Attempts to write more than size bytes to the buffer result in an error.

Acorn
  • 24,970
  • 5
  • 40
  • 69
  • Here is a library that looks promising: https://github.com/Arryboom/fmemopen_windows – Justin Meiners Oct 10 '22 at 19:32
  • I recommend also setting `FILE_FLAG_WRITE_THROUGH` and `FILE_FLAG_NO_BUFFERING` to avoid caching behaviour that would be completely redundant in this context. – Zaaier Jun 15 '23 at 21:26
1

You could achieve something close with MapViewOfFile. Win32 API you should use for this are:

CreateFile(...)

CreateFileMapping(...)

MapViewOfFile(...)

Don't forget to unmap and close file mapping once you are done.

Community
  • 1
  • 1
Indy9000
  • 8,651
  • 2
  • 32
  • 37
  • are there functions available for plain C console app? – J Nash Sep 03 '12 at 14:37
  • Win32 is a plain c API. You would need the Windows SDK installed and make sure to include the correct header files specified on the MSDN pages I linked – Indy9000 Sep 03 '12 at 14:39
  • 1
    To map a file into virtual memory you don't need the `CreateFile`, use a file handle of `INVALID_HANDLE_VALUE` (-1) in `CreateFileMapping`. – cdarke Sep 03 '12 at 14:59
  • 20
    This accomplishes the opposite goal. `Fmemopen` takes a memory buffer and lets you treat it like a file — it returns a `FILE*`. `MapViewOfFile` takes a file (mapping) and lets you treat it like a memory buffer — it returns a `void*`. – Rob Kennedy Sep 23 '14 at 17:23
  • 2
    @RobKennedy This single comment has more upvotes than anything else on this question. Please write an answer. Once you've done so, you can ping me for a free upvote. – wizzwizz4 Jan 14 '17 at 15:56
  • 3
    Thank you, @Wizz, but my comment doesn't answer this question, so I don't really have anything to post. – Rob Kennedy Jan 14 '17 at 17:05
0

I would not use fopen as this creates a file.

To create the file in memory you will need to create the file and store it in some way that you see fit.

You will need to create an array of some datatype. I would use char* but you could use whatever you want to encode the imformation. You can then read and write to the file by adding information to the array.

KennyBartMan
  • 940
  • 9
  • 21
  • Thanks for reply, my only problem is that I was told to use external program that uses *FILE, so I'm forced to use it. Second request was to not write anything to disk, I'm searching for some kind of workaround e.g.virtual file in memory, memory mapping etc. – J Nash Sep 03 '12 at 14:30
  • @J Nash: FILE pointers are entirely implementation defined and not portable across compilers. If the external program itself FILE internally I cannot see how that would affect your program. – Clifford Sep 03 '12 at 14:51
  • 1
    Sorry I didn't explain thoroughly, I'm adding encryption into existing program, which is quite complex. It reads input files existing on disk.Currently I'm encrypting those files first, and decrypt back inside my program- however it creates temporary "open unecnrypted" files which is not secure solution. Thanks – J Nash Sep 03 '12 at 15:00
  • 1
    @J Nash: do you perhaps want to use a pipe? – cdarke Sep 03 '12 at 15:00
0

Such functionality is not intrinsically supported in Windows. However you could rewrite the high-level stdio functions to access a memory buffer rather than a file, or just implement stdio-like functions. Or you could implement a RAM disk, however installing the RAM disk driver itself may not meet your "do not write to the disk" rule.

Clifford
  • 88,407
  • 13
  • 85
  • 165