How to read from pipe into a file?

Question

I would like to read from the pipe straight into a file with the code below. base_fd is a pipe.

FILE* fp = fopen("dec_data", "wb+"); 
int r_result; 
int len = msg_length-part-3;  //set to 75933
while ((r_result = read(base_fd[0], fp, len))) {
       printf("r_result: %d \n", r_result);
       len -= r_result; 
       }

The read seems to happen fine, with r_result showing 65536 and then 10397 as required. However, when I inspect the file I created, it has a size of 0 bytes...

printf directs output to stdout. I think you're looking for fprintf — Fubar, Apr 17 '20 at 19:58
The printf is just for error checking. The read call is supposed to write the file, or at least that was the aim — Cheetaiean, Apr 17 '20 at 19:59
oh... missed that. Not sure you can use a file pointer as a buffer. You're probably telling C to put the content you read into the address where the file pointer is stored. — Fubar, Apr 17 '20 at 20:03
The second argument to `read()` should be a pointer to a buffer that will hold the data you're reading. A `FILE *` is not going to work there. You need to `read()` into a buffer, then `write()` from the buffer to the file. — Jim Lewis, Apr 17 '20 at 20:04
Unfortunate, but makes sense. Thank you. What is interesting is that no errors are thrown up - the program permits the operation. — Cheetaiean, Apr 17 '20 at 20:17
The second arg of read is `void *` meaning you can throw any pointer in there, regardless whether it does anything sensible. — stark, Apr 17 '20 at 20:22

score 0 · Accepted Answer · answered Apr 17 '20 at 21:04

You have a semantic error in your code. Take a look at the read(2) system call signature:

#include <unistd.h>

ssize_t read(int fd, void *buf, size_t count);

The second parameter to the function is a void pointer (void *buf), which is where read will store the count bytes it reads from fd descriptor.

However, a FILE * is an abstraction of the C library. In this answer you can see more of it. The struct FILE in MinGW32 5.1.4 is:

typedef struct _iobuf
{
    char*   _ptr;
    int _cnt;
    char*   _base;
    int _flag;
    int _file;
    int _charbuf;
    int _bufsiz;
    char*   _tmpfname;
} FILE;

What read will do is similar to how we copy strings. Consider this function:

void strcpy(char *dst, char *src)
{
    while(*src) *dst++ = *src++;
}

This function will copy the contents from src into dst until it finds a NULL terminating byte. This is obviously a very flawed function and should never be used, but illustrates why your example doesn't work.

Under the hood, what read is doing is very similar to this strcpy function: it is overwriting a lot of bytes in memory starting at the address pointed to by the fp pointer. You are effectively losing your reference to the FILE * pointer and the resources associated to it.

I'll bet that if you try to close(fp) after that loop you'll get a segmentation fault (it's Undefined Behavior, but I'll bet anyway).

The right way to do what you want is:

FILE* fp = fopen("dec_data", "wb+"); 
char *buf;
int r_result; 
int len = msg_length - part - 3;  //set to 75933

buf = malloc(len);
if(!buf) {
    perror("malloc");
    exit(EXIT_FAILURE);
}

while ((r_result = read(base_fd[0], buf, len))) {
    fprintf(fp, buf);
    len -= r_result; 
}

free(buf);
close(fp); // now it closes the file pointer

If we could access the internals of struct _iobuf, perhaps the char * _ptr or base, maybe it would work if we had a read(base_fd[0], fp->_ptr, len) — Cheetaiean, Apr 17 '20 at 23:44
@Cheetaiean Probably not. The standard library buffers the IO, so it keeps track of how much it writes/reads from/to the file. If you write directly to the pointer you won't update housekeeping data, which will case the lib to fail anyway. If you really need to read directly into another file _using a pointer_, maybe [mmap(2)](http://man7.org/linux/man-pages/man2/mmap.2.html) helps you. — Enzo Ferber, Apr 18 '20 at 01:18

How to read from pipe into a file?

1 Answers1