0

I have this code snippet to write a buffer to a file

int WriteBufferToFile(std::string path, const char* buffer, int bufferSize) {
    std::ofstream ofs;
    ofs.open(path);

    if (!ofs) {
        return 1;
    }

    ofs.write(buffer, bufferSize);    

    if (!ofs) {
        return 2;
    }

    ofs.close();

    return 0;
}

Case 1

std::vector<char> buffer(1000000, 0);

WriteBufferToFile("myRawData", buffer.data(), 1000000);

Case 2

std::vector<char> buffer(1000000);

for (int i = 0; i < 1000000; i++) {
    buffer[i] = char(i);
}

WriteBufferToFile("myRawData2", buffer.data(), 1000000);

In Case 1 one I'm writing 1mb of just zeros to a file, which also will have 1mb in size, but in the second case i write arbitary chars (still should be 1mb in RAM) to a file, but now (in my tests it seems like especially when char's >= 10 are contained) the file size increases.

enter image description here

Why is that, and is there a way to fix this?

Cody Gray - on strike
  • 239,200
  • 50
  • 490
  • 574
T. Grumser
  • 99
  • 3
  • 18

2 Answers2

8

I'm going to take a wild guess and say that you're running this code on a Windows system.

Here's what I think is probably happening.

ofs.open(path) is opening the file in text mode. On Windows, text mode means that every newline character (1 byte) will be replaced by a CRLF sequence (2 bytes). Your buffer contains 1 million characters which are filled with the values 0 to 999999 modulo 256. So 1 on 256 characters (3906 to be exact) will be replaced by a 2 byte sequence which accounts for the file size difference.

To fix this, open the file in binary mode:

ofs.open(path, ios_base::out | ios_base::binary)
Ferruccio
  • 98,941
  • 38
  • 226
  • 299
3

You seem to be on Windows. It looks like the C++ runtime rewrites 1-byte linebreak characters '\n' to the windows style 2-byte sequence "\r\n". Therefore, you have about 1000000/256=3906.25 additional bytes in the file.

j6t
  • 9,150
  • 1
  • 15
  • 35