fwrite() adds garbage data to output (WINE & Windows 7, mingw & MSVC; NOT linux/gcc)

Question

Under certain circumstances, fwrite writes extra data (more bytes than requested). The output of a short demo is the easiest way to explain. The demo attempts to create two 2048 byte files and checks the offset after each fwrite call to determine the number of bytes written. The first fwrite call writes two extra bytes:

len: 2048
current offset = 0
wrote 1024 bytes
current offset = 1026
EXITING:
offset % BLOCKSIZE = 2

len: 2048
current offset = 0
wrote 1024 bytes
current offset = 1024
wrote 1024 bytes
SUCCESS

The program runs successfully (writes 2048 bytes to both files) when compiled as an ELF (unix binary), but fails (as shown above) when compiled as a PE (windows binary/executable). I've tried compiling and testing with:

Ubuntu 14.04 and gcc 4.8.2 - SUCCESS
WINE 1.6.2 and mingw 4.8.2 - FAIL
Windows 7 and mingw 4.8.2 - FAIL
Windows 7 and Visual Studio 2013 - FAIL

The actual data in the buffer passed to fwrite affects the number of extra bytes written, but it happens virtually every time (unless you're writing NULL bytes).

main.c:

#include <stdio.h>

#include "stub.h"
#include "stub2.h"

size_t min(size_t a, size_t b)
{
    return a<b?a:b;
}

#define BLOCKSIZE 1024

void test(char buf[], size_t len)
{
    FILE *f = fopen("out", "w");

    printf("len: %lu\n", len);
    for(size_t i=0;i<len;i+=BLOCKSIZE)
    {
        printf("current offset = %lu\n", ftell(f));
        if(ftell(f) % BLOCKSIZE)
        {
            printf("EXITING:\noffset %% BLOCKSIZE = %d\n\n", ftell(f) % BLOCKSIZE);
            return;
        }
        size_t wrote = fwrite(buf + i, 1, min(len - i, BLOCKSIZE), f);
        printf("wrote %lu bytes\n", wrote);
    }

    printf("SUCCESS\n\n");

    fclose(f);
}

int main()
{
    test(stub_exe, stub_exe_len);
    test(stub2_exe, stub2_exe_len);
    return 0;
}

stub.h and stub2.h are generated from the 2048 bytes of /dev/urandom and 2048 bytes from /dev/zero (respectively) with xxd. For example:

dd if=/dev/urandom of=stub2.exe bs=2048 count=1
xxd -i stub.exe stub.h
dd if=/dev/zero of=stub2.exe bs=2048 count=1
xxd -i stub2.exe stub2.h

On Windows if a file is not opened in binary mode any `\n` characters will be written as `\r\n` which is probably the difference you're seeing. — Retired Ninja, Aug 21 '14 at 07:09
If you are using Windows/Cygwin you are expected to select binary mode to the fopen call (put "wb" instead of "w" as second parameter to fopen) or you'll get \n expanded as \r\n on output. I have not been able to reproduce this behaviour in linux, as it always opens files in binary mode. — Luis Colorado, Aug 22 '14 at 05:36
possible duplicate of [End of FILE\* pointer is not equal to size of written data](http://stackoverflow.com/questions/145006/end-of-file-pointer-is-not-equal-to-size-of-written-data) — Raymond Chen, Aug 27 '14 at 03:16

score 5 · Accepted Answer · edited Jun 20 '20 at 09:12

Disclaimer: It's been 5 days and no one has posted an "official" answer, so I'm making this answer and accepting it. Credit goes to Christophe and Retired Ninja for actually answering the question.

The simple answer is that you need to open binary files in binary mode (by adding a 'b' to `fopen`'s 2nd argument).

By default, fopen opens files in text mode, rather than binary mode. In text mode, the newline character (\n) is replaced by the operating system's newline sequence when writing to the file, and vice versa when reading. Unlike POSIX systems (Linux, OSX, BSD, etc), which use a single character (\n) as the newline, Windows uses the \r\n newline sequence.

fwrite() adds garbage data to output (WINE & Windows 7, mingw & MSVC; NOT linux/gcc)

1 Answers1

The simple answer is that you need to open binary files in binary mode (by adding a 'b' to fopen's 2nd argument).

The simple answer is that you need to open binary files in binary mode (by adding a 'b' to `fopen`'s 2nd argument).