Synchronisation issue calling ofstream write with custom buffer (pubsetbuf)

Question

I am writing potentially large (multi-gigabyte) binary files to a network location using std::ofstream. Previously I was flushing the buffer at frequent intervals, which hurt performance especially at peak times - reducing the flush frequency has helped. I am interested in finding other ways to further improve write performance.

According to various online sources, writing directly to a buffer associated with the stream's streambuf object via stream.rdbuf()->pubsetbuf(buffer) avoids the write overhead described here.

Instead of passing each value in turn to ofstream::write, I populate and pass a byte buffer which has a fixed size of 8192 bytes. I had expected that directly populating the buffer would avoid the need to call write, however, omitting write results in nothing being written to file. In any case, my tests show that the use of pubsetbuf with write is significantly faster than not using pubsetbuf and writing one 8-byte value at a time.

Unfortunately, if I do not flush the stream following each write call, the content of the buffer is both appended to the file AND written at the start, overwriting the data previously at that position. If I flush immediately following the write call, the resulting file is correct. I have checked that the stream pointer is correct every time a write happens.

Alternatively, if I don't bother calling stream.rdbuf()->pubsetbuf(buffer) at all and continue to pass the pre-populated byte buffer to write, I avoid the need to flush. I'm currently profiling which of the two approaches performs better:

• pubsetbuf with write + flush every time (suspect poor performance over network)
• no pubsetbuf, write and a single flush at the end

This illustrates the overwrite issue with a much reduced data set (comment stream.flush() back in to have it behave correctly):

vector<double> data;
for (int i = 0; i < 400; ++i) data.push_back(111.0);
for (int i = 0; i < 400; ++i) data.push_back(222.0);
for (int i = 0; i < 400; ++i) data.push_back(333.0);

bool lastPoint;
size_t numPoints = data.size();
size_t bytesPerValue = sizeof(data[0]);

// Create a byte buffer
static const size_t BUFFER_SIZE = 8192;
vector<unsigned char> buffer;
buffer.resize(BUFFER_SIZE);
size_t bufferIndex = 0;
bool bufferFull = false;

// Create output stream
ofstream stream("\\\\server\\networkpath\\output.dat", ios::binary);

// Set stream buffer to avoid write calls
stream.rdbuf()->pubsetbuf(reinterpret_cast<char*>(&buffer[0]), BUFFER_SIZE);

for (size_t i = 0; i < numPoints; i++)
{
    lastPoint = i == (numPoints - 1);

    // Write to buffer
    memcpy(&buffer[bufferIndex], reinterpret_cast<const unsigned char*>(&data[i]), bytesPerValue);

    // Find next empty index
    bufferIndex += bytesPerValue;

    bufferFull = ((bufferIndex + bytesPerValue) > BUFFER_SIZE);

    if (lastPoint || bufferFull)
    {
        stream.write(reinterpret_cast<const char*>(&buffer[0]), bufferIndex);

        //stream.flush();

        // If the line above is commented back in, the content of the output file is correct.
        // However, I wish to avoid flushing often when writing across network.
        // Without commenting back in, the file contains the expected number of bytes (9600) but the data in the second write call 
        // (bytes 8193-9600) appears both at the start and end of the file.

        bufferIndex = 0;
    }
}
stream.flush();
stream.close();

Is there a way to use the buffering technique via pubsetbuf without the need to flush every time a write is performed?