Send file writes to in-memory buffer (fopen something, but write to buffer, not disk)

Question

I'd like to:

redirect stdout/stderr
to an in-memory buffer, rather than to disk
while leaving stdout/err running as normal

I know how to use dup2 and freopen to:

redirect stdout/err to a file of my choosing
while leaving stdout/err running as normal

...but I'm not sure about item 2?

REASON: I want to post-process the data going to stdout/err (that's coming from 3rd party code), and then send it to a webserver - while leaving the application running as normal.

WHY? ... because right now when our app runs we're seeing critical data go to stdout (and err) from 3rd party code, and it's a PITA getting users to try and mess around with local logfiles. I'd rather capture "the last N bytes" into a ring buffer, post-process it, and then - iff there's a problem that the user reports - send that to server.

WHY AVOID FILES? ... because this is for code that has to run on iOS as well as desktop, and "constantly writing to a file" is something I want to avoid, since I don't want it as a file anyway. Also ... I'd have to worry about maintaining that file, proactively trimming its size, etc.

You could use a pipe. Check out this solution: http://stackoverflow.com/questions/955962/how-to-buffer-stdout-in-memory-and-write-it-from-a-dedicated-thread — arc, Mar 06 '12 at 20:22
Some Unixes or Posixes have the http://pubs.opengroup.org/onlinepubs/9699919799/functions/fmemopen.html function (inspired by GNU). GNU also have http://linux.die.net/man/3/open_memstream perhaps your IOS have these. — Basile Starynkevitch, Mar 06 '12 at 20:22
Why is it that people seem to have no idea what a unix socket is? Create one, dup2 your 3rd-party stdin/stdout to it, use your code to post-process it, no disk involved. — tbert, Mar 06 '12 at 20:37
@arc I like the idea of using Pipes, but according to that question, pipes can block, and performance can nosedive. Not sure, but I'm going to try with memory streams first, seems like that's closer to what C wants me to do. — Adam, Mar 06 '12 at 23:08
@JanX2 look at Spencer's answer below - the project I'm using it on got shelved, so I never got to confirm it works, but it looked to me like it would work fine. (and ... if Spencer's approach works, please let us know and I can mark the answer ACCEPTED) — Adam, Jan 22 '13 at 21:30
https://stackoverflow.com/questions/539537/how-to-write-to-a-memory-buffer-with-a-file — Ciro Santilli OurBigBook.com, Sep 20 '17 at 06:44

score 7 · Accepted Answer · edited Feb 23 '18 at 17:49

Stdout is a basic file handle, which means its a pointer.

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv);

int main (argc,argv)
int argc;
char **argv;
{
    stdout = &(*stderr);
    fprintf(stdout, "test");
    return 0;
}

By declaring stdout to point to a different file handle, this program redirects stdout. To test it, compile it as test, and then type ./test > t and ./test 2> t on the command line.

The redirected stdout pipe will still output the word test, whereas the redirected stderr pipe will not.

Since files are streams, read this section of the GNU C Library for how to create memory streams. This example creates a text buffer to dynamically resize for holding input, and redirects stdout to point to it.

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv);

int main (argc,argv)
int argc;
char **argv;
{
    char *bp;
    size_t size;
    FILE *stream;

    stream = open_memstream (&bp, &size);
    stdout = &(*stream);
    fprintf(stdout, "test");
    fflush(stdout);
    fprintf(stderr, "buf = %s, size = %d\n", bp, size);
    fclose (stream);
    free(bp);
    return 0;
}

Pay careful attention to the sections on flushing the buffers when switching between input and output in the manual.

Ah, thanks - seems "memory streams" was the magic phrase I needed to google. I'll try this approach with iOS and see if it works. Fingers crossed... — Adam, Mar 06 '12 at 23:09
I suggest adding a `free(bp)` to the end. It doesn't matter here, but if someone is cutting and pasting this into a function, they'll need it. — Mark Lodato, Nov 19 '14 at 23:03
I never finished testing this, but I'm marking as accepted anyway - it looks right to me, and Spencer should get credit. — Adam, Jan 30 '16 at 19:45

Send file writes to in-memory buffer (fopen something, but write to buffer, not disk)

1 Answers1