fwrite() - effect of size and count on performance

Question

There seems to be a lot of confusion regarding the purpose of the two arguments 'size' and 'count' in fwrite(). I am trying to figure out which will be faster -

fwrite(source, 1, 50000, destination);

or

fwrite(source, 50000, 1, destination);

This is an important decision in my code as this command will be executed millions of times.

Now, I could just jump to testing and use the one which gives better results, but the problem is that the code is intended for MANY platforms.

So,

How can I get a definitive answer to which is better across platforms?
Will implementation logic of fwrite() vary from platform to platform?

I realize there are similar questions (What is the rationale for fread/fwrite taking size and count as arguments?, Performance of fwrite and write size) but do understand that this is a different question regarding the same issue. The answers in similar questions do not suffice in this case.

I just did some tests on OS-X, writing ten 100MB files, and there was no difference between the order of parameters, or when using write(2) instead of fwrite. As for other platforms, I can't say. — William Morris, May 12 '12 at 14:55

Jens · Accepted Answer · 2012-05-16T10:04:16.247

The performance should not depend on either way, because anyone implementing fwrite would multiply size and count to determine how much I/O to do.

This is exemplified by FreeBSD's libc implementation of fwrite.c, which in its entirety reads (include directives elided):

/*
 * Write `count' objects (each size `size') from memory to the given file.
 * Return the number of whole objects written.
 */
size_t
fwrite(buf, size, count, fp)
    const void * __restrict buf;
    size_t size, count;
    FILE * __restrict fp;
{
    size_t n;
    struct __suio uio;
    struct __siov iov;

    /*
     * ANSI and SUSv2 require a return value of 0 if size or count are 0.
     */
    if ((count == 0) || (size == 0))
        return (0);

    /*
     * Check for integer overflow.  As an optimization, first check that
     * at least one of {count, size} is at least 2^16, since if both
     * values are less than that, their product can't possible overflow
     * (size_t is always at least 32 bits on FreeBSD).
     */
    if (((count | size) > 0xFFFF) &&
        (count > SIZE_MAX / size)) {
        errno = EINVAL;
        fp->_flags |= __SERR;
        return (0);
    }

    n = count * size;

    iov.iov_base = (void *)buf;
    uio.uio_resid = iov.iov_len = n;
    uio.uio_iov = &iov;
    uio.uio_iovcnt = 1;

    FLOCKFILE(fp);
    ORIENT(fp, -1);
    /*
     * The usual case is success (__sfvwrite returns 0);
     * skip the divide if this happens, since divides are
     * generally slow and since this occurs whenever size==0.
     */
    if (__sfvwrite(fp, &uio) != 0)
        count = (n - uio.uio_resid) / size;
    FUNLOCKFILE(fp);
    return (count);
}

I ran tests on several platforms and looked at the source code for fwrite for some others, no performance difference in any of them. Thanks! — Shailesh Tainwala, May 16 '12 at 09:06

score 15 · Answer 2 · edited Mar 30 '15 at 20:00

15

The purpose of two arguments gets more clear, if you consider ther return value, which is the count of objects successfuly written/read to/from the stream:

fwrite(src, 1, 50000, dst); // will return 50000
fwrite(src, 50000, 1, dst); // will return 1

The speed might be implementation dependent although, I don't expect any considerable difference.

edited Mar 30 '15 at 20:00

Antoine

3,880
2
26
44

answered May 12 '12 at 15:01

Aleš Kotnik

2,654
20
17

3

Agreed. In particular, the first will fail if it only get 49999 bytes (returning 0); the second will return 49999 if that's the number of bytes it reads. So, if your data is rigidly 50,000 bytes, it matters not in the least which you use. If your data is not rigid but might be shorter, then you need the more flexible option. – Jonathan Leffler May 12 '12 at 18:14
I mistakenly use `fread` instead of `fwrite` but the same principle applies eitherway. – Aleš Kotnik May 13 '12 at 09:08

score 2 · Answer 3 · edited May 23 '17 at 11:46

I'd like to point you to my question, which ended up exposing an interesting performance difference between calling fwrite once and calling fwrite multiple times to write a file "in chunks".

My problem was that there's a bug in Microsoft's implementation of fwrite so files larger than 4GB cannot be written in one call (it hangs at fwrite). So I had to work around this by writing the file in chunks, calling fwrite in a loop until the data was completely written. I found that this latter method always returns faster than the single fwrite call.

I'm in Windows 7 x64 with 32 GB of RAM, which makes write caching pretty aggressive.

fwrite() - effect of size and count on performance

3 Answers3

Linked

Related