16

If I have a buffer which contains the data of a file, how can I get a file descriptor from it? This is a question derived from how to untar file in memory

Community
  • 1
  • 1
solotim
  • 1,826
  • 4
  • 24
  • 41

3 Answers3

16

I wrote a simple example how to make filedescriptor to a memory area:

#include <unistd.h>
#include <stdio.h> 
#include <string.h> 

char buff[]="qwer\nasdf\n";

int main(){
  int p[2]; pipe(p);

  if( !fork() ){
    for( int buffsize=strlen(buff), len=0; buffsize>len; )
      len+=write( p[1], buff+len, buffsize-len );
    return 0;
  }

  close(p[1]);
  FILE *f = fdopen( p[0], "r" );
  char buff[100];
  while( fgets(buff,100,f) ){
    printf("from child: '%s'\n", buff );
  }
  puts("");
}
sambowry
  • 2,436
  • 1
  • 16
  • 13
  • Nice. As a modest enhancement, you could use [`fcntl(fd, F_SETPIPE_SZ, ...)`](http://www.kernel.org/doc/man-pages/online/pages/man2/fcntl.2.html) to increase the pipe buffer size. Credit to [msandiford](http://stackoverflow.com/a/14129599/781723) for this. – D.W. Jun 19 '15 at 00:41
  • 2
    On Linux, [`vmsplice()`](http://man7.org/linux/man-pages/man2/vmsplice.2.html) might be useful: it avoids the need for a for-loop to write data to the pipe. – D.W. Jun 19 '15 at 00:59
  • 1
    This is a pretty awesome solution, but if you need to be able to do more than read from the fd (i.e. seek), then this won't work. You cannot seek on a pipe (at least not on the platforms I use). – John Bowers Jun 09 '16 at 19:37
  • 2
    Why didn't you choose `fmemopen()`? Yes, it's POSIX and not C, but `pipe()` is also POSIX, isn't it? – Mecki Jan 17 '19 at 16:47
  • 4
    @Mecki it appears from the man pages that `fileno()` will return `-1` when called on a `FILE*` returned from `fmemopen()`. – tekknolagi Jul 29 '19 at 18:01
  • 1
    Just to provide a citation for @tekknolagi's comment, `man fmemopen` says "There is no file descriptor associated with the file stream returned by this function (i.e., fileno(3) will return an error if called on the returned stream)." It doesn't say why it doesn't work, though. I'm a bit bummed it doesn't. – Faheem Mitha Oct 26 '21 at 18:50
  • @FaheemMitha file descriptors are references to "files" managed by the operating system. The OS doesn't have the introspection into how you use different parts of memory in your process to know that you're using some chunk of memory as a "file" managed by the C library/runtime. – mtraceur May 13 '22 at 02:09
5

Not possible in plain C. In plain C all file access happens via FILE * handles and these can only be created with fopen() and freopen() and in both cases must refer to a file path. As C tries to be as portable as possible, it limits I/O to the absolute bare minimum that probably all systems can support in some way.

If you have POSIX API available (e.g. Linux, macOS, iOS, FreeBSD, most other UNIX systems), you can use fmemopen():

char dataInMemory[] = "This is some data in memory";
FILE * fileDescriptor = fmemopen(dataInMemory, sizeof(dataInMemory), "r");

This is a true file handle that can be used with all C file API. It should also allow seeking, something not possible if you work with pipes as pipes support no seeking (you can emulate forward seeking but there is no way to ever seek backwards).

Mecki
  • 125,244
  • 33
  • 244
  • 253
-1

You can't. Unlike C++, the C model of file I/O isn't open to extension.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • 2
    fmemopen can return FILE* from buffer, but fileno(fmemopen(...)) return -1. I got another idea: create pipe and feed buffer content to file_pipes[1] by write() function, and we can look the file_pipes[0] as the file descriptor of that buffer. But when I practise this, the write() function just blocked. Is the kernel buffer of pipe not big enough? Thanks – solotim Oct 13 '09 at 08:12
  • 1
    That's POSIX, IIRC. Not C, which is how you tagged your question. I.e. it wouldn't work on Windows. – MSalters Oct 13 '09 at 08:39