0

I have written code to read a big size file content and write that content into a new file.

That code works fine with a small and medium size file content but with a big size file content, approximately 1.8GB and above it does not works and gives me an unknown error/exception during runtime.

Also, I have tried to debug and the following is the debugging result:

The code:

char * myClass::getFileContent(const char * fileName) {
    std::ifstream file(fileName, std::ios::binary|std::ios::ate);
    if (!file.is_open()) {
        perror(strerror(errno));
        return "";
    }
    char * strBuffer = NULL;
    long long length = file.tellg();
    strBuffer = new char[length];
    file.seekg(0, std::ios::beg);
    file.read(strBuffer, length);
    return strBuffer;
}

// The implementation
char *fileName = "C:\\desktop-amd64.iso";
char *fileContent = myClass.getFileContent(fileName);
ofstream file("c:\\file.iso", ios::binary);
if (file.is_open()) {
    file.write(fileContent, myClass.getFileSize(fileName));
    file.close();
}
delete fileContent;

Note: I am using Visual Studio 2015 on Windows 7 x64.

Why does that problem happen with the big files?

Lion King
  • 32,851
  • 25
  • 81
  • 143
  • 1
    Are you by any chance on a 32 bit system or using a 32 bit compiler? – n. m. could be an AI Apr 29 '19 at 13:29
  • 5
    `free(fileContent)` - you're using `free` on a variable created by `new`. That's certainly wrong. You should be using `delete`. – Chris White Apr 29 '19 at 13:30
  • 1
    On a possibly related note, you are writing in C++ but using mostly C idioms with some C++ APIs thrown in here and there. This often results in clunky, fragile, buggy code. By the way do you really need to read the entire file contents at once? – n. m. could be an AI Apr 29 '19 at 13:34
  • @ChrisWhite: I forgot to use delete instead free, but I have changed and the problem still occurs. – Lion King Apr 29 '19 at 13:38
  • @n.m.: I think you are right, I was using 32-bit compiler version, but after changing to x64 compiler version the runtime error stopped. Is that the problem? – Lion King Apr 29 '19 at 13:47
  • You obviously cannot load a big file to a small RAM all at once, so it is unclear what you expected from this. – n. m. could be an AI Apr 29 '19 at 13:53
  • @n.m.: So, what's the optimal alternative way to do that? – Lion King Apr 29 '19 at 14:00
  • Don't attempt to load a big file to a small RAM all at once since this is clearly impossible. Anyway it looks like switching to a 64 bit compiler has solved your problem. – n. m. could be an AI Apr 29 '19 at 14:24

2 Answers2

3

The debugger shows that an std::bad_alloc exception occurs at the line

strBuffer = new char[length];

It seems that you are trying to allocate a block in memory that's size is the same as the file you are trying to read. As the file's size is around 1.8 GB, it is possible that the operating system just cannot allocate a chunk in memory with that size.

I recommend reading this answer about handling huge files without storing all their contents in the memory.

Tr3bron
  • 31
  • 1
  • 2
0

As far i can see. You are trying to release memory with free while you allocate memory with new. You cannot mix them.

To deallocate memory, use delete.

Now, if you need to free it with free, you must allocate memory with malloc.

Jordan Motta
  • 188
  • 1
  • 5