emplace_back + ofstream + windows -> bug?

Question

The code below tries to create 800 files in the ./out folder. On Linux Debian with g++ 4.9.2 the program works correctly. But on Windows 7 or 8 with g++ 5.2.0 (MinGW) the program stops at 509 files. The error seems to be a combination of using emplace_back and ofstream in a constructor of class Task. Is it bug?

//g++ -std=c++11 main.cpp
#include <sstream>
#include <fstream>
#include <list>
#include <iostream>

#ifdef _WIN32
    #include <io.h>
#else
    #include <sys/stat.h>
#endif

using namespace std;

int i;

struct Task
{
    ofstream out;
    Task(string file_name): out(file_name)
    {
        if(!out) {cout<<i<<"\n"; exit(1);}
    }
};

int main()
{       
    #ifdef _WIN32
        string output_folder = ".\\out";
        mkdir(output_folder.c_str());
        output_folder+="\\";
    #else
        string output_folder = "./out";
        mkdir(output_folder.c_str(),S_IRWXU);
        output_folder+="/";
    #endif

    list<Task> ltask;
    for(i=0; i<800; i++)
    {
        ostringstream os;
        os<<output_folder<<i;
        ltask.emplace_back(os.str());
    }
    return 0;
}

If you are running antivirus on the windows machine I would turn it off and try again. It could be that it is stopping it from creating too many files. — NathanOliver, Dec 18 '15 at 20:42
Is this a FAT16 filesystem by any chance? In that case [the maximum number of files in a single folder is 512](http://stackoverflow.com/a/14407078/5181033) — Carsten Hansen, Dec 18 '15 at 20:53

score 3 · Answer 1 · answered Dec 18 '15 at 20:57

3

It is a limitation on the number of opened files (in Windows or runtime libraries or somewhere else)... Changing the cycle in the code above with the following code leads to the same result:

list<ofstream*> l;
for(i=0; i<800; i++)
{
    ostringstream os;
    os<<output_folder<<i;
    l.push_back(new ofstream(os.str()));
}
for(auto p: l) delete p;

answered Dec 18 '15 at 20:57

Sergey

41
4

So if he closed the file after writing, he would be Ok? – Robert Jacobs Dec 18 '15 at 21:06
1

The problem is the number of simultaneously opened files. It can not be larger than 512 per process in Windows. The above example with "delete" shows that emplace_back was suspected wrongly. – Sergey Dec 18 '15 at 21:24
@Sergey You are right, I deleted the post. Thanks for your comment. – KRoy Jan 22 '18 at 19:50

score 0 · Answer 2 · answered Jan 10 '18 at 00:00

See this question.

The CRT (which is used by the C++ Standard Library) has, by default, a limit of 512 simultaneously open file descriptors. You're using 3 for stdin, stdout, and stderr, which leaves 509. (Your Tasks aren't just creating the files; they're keeping them open.)

If you close the files after creating them, you'll be fine. You can also raise the limit (explained in the other question) or use Windows file handles, of which there's a much higher limit.

emplace_back + ofstream + windows -> bug?

2 Answers2