Make a file pointer read/write to an in-memory location

Question

I can make a file pointer write to a file with fopen(). But can I make a file pointer that will make it so calling functions such as fputc or fprintf will write to a pointer in memory? An example of this is ByteArrayOutputStream in java. Also: could I run it in reverse, where a library needs a file pointer to read from, so I allocate memory, and make a new file pointer that will read from this memory location but return EOF when the size of the chunk runs out? (like ByteArrayInputStream in Java). Is there a way to do this in C? For example:

FILE *p = new_memory_file_pointer();
fprintf(p, "Hello World!\n");
char *data = get_written_stuff(p);
printf("%s", data); //will print Hello World!

&& / ||

char s[] = "Hello World!\n";
FILE *p = new_memory_file_pointer_read(s, sizeof(s));
char *buffer = (char *)malloc( 1024*sizeof(char) );
fread((void *)buffer, 1, sizeof(s), p);
printf("%s", buffer); //prints Hello World!

EDIT: To those reading this question years later, in addition to the accepted answer, you should look at open_memstream(3), which behaves more like these Java classes than fmemopen does.

Answer 1

If your operating system provides fmemopen, probably it will meet your purpose.

Answer 2

In C++ (and you added the C++ tag) you can write a function accepting an arbitrary input/output stream. Since std::stringstream or std::ofstream are derived classes you can pass both of them equally into this function. An example:

#include <iostream> // for std::cout
#include <fstream> // for std::ofstream
#include <sstream> // for std::stringstream

void write_something(std::ostream& stream) {
  stream << "Hello World!" << std::endl;
}

int main() {
  write_something(std::cout); // write it to the screen
  {
    std::ofstream file("myfile.txt");
    write_something(file); // write it into myfile.txt
  }
  {
    std::stringstream stream;
    write_something(stream); // write it into a string
    std::cout << stream.str() << std::endl; // and how to get its content
  }
}

And analogously with std::istream instead of std::ostream if you want to read the data:

void read_into_buffer(std::istream& stream, char* buffer, int length) {
  stream.read(buffer, length);
}

int main() {
  char* buffer = new char[255];

  {
    std::ifstream file("myfile.txt");
    read_into_buffer(file, buffer, 10); // reads 10 bytes from the file
  }
  {
    std::string s("Some very long and useless message and ...");
    std::stringstream stream(s);
    read_into_buffer(stream, buffer, 10); // reads 10 bytes from the string
  }
}

Answer 3

As Ise aptly points out, there is a function for this fmemopen in POSIX 2008 and that is supported on Linux. Using POSIX 2004, probably the closest thing to that would be to use a temporary file:

// This is for illustration purposes only. You should add error checking to this.
char name[] = "/tmp/name-of-app-XXXXXX";
int temporary_descriptor = mkstemp(name);
unlink(temporary_descriptor);
FILE* file_pointer = fdopen(temporary_descriptor, "r+");
// use file_pointer ...

It's easier to do the reverse; that is, to have a function that writes into memory, and then to use that both to write into memory and also to write to a file. You can use mmap so that a chunk of memory ends up being backed by a file, and writing into that memory writes the data to the associated file.

If you use std::istream and std::ostream instead of low-level C FILE* objects, then C++ conveniently provides std::istringstream and std::ostringstream for reading/writing strings.

You will note that almost all of the functions in C that begin with an "f" and that operate on files, have equivalents beginning with "s" that operate on strings. A perhaps better approach would be to design an interface for I/O that is not specific to either files or strings, and then provide implementations that connect to files and strings, respectively. Then implement your core logic in terms of this interface, instead of in terms of low-level C and C++ I/O. Doing that also has the benefit of allowing for future extensions, such as supporting network files with builtin support for compression, duplication, etc.

Answer 4

With a memory mapped file. This is platform specific so you'll need to find the information about creating mem-mapped files on your target system(s). I believe the posix version is mmap. At any rate, searching for "memory mapped file" on google should turn up a bunch of help.