fopen does not deal with more than 60 files at the same time

Question

I need to have more than 60 text files opened at the same time in my C program. However, it seems that fopen is not able to handle more than 60 files simultaneously. I am programming in Windows environment.

I use the following fopen statement:

fopen(fileName.c_str(),"wt");

Where fileName is the path of my txt file, name which changes inside a loop along 100 files. Does anybody know any trick to make this work? Or any alternative?

First of all you might want to check `errno` when `fopen` fails. Also, I guess you are on Windows? — Some programmer dude, Sep 04 '13 at 09:03
Any reason why you cannot have only a few recently used files open? Then close old ones and open new ones when necessary. — user694733, Sep 04 '13 at 09:07
user694733, my algorithm needs to have all these files opened, so I cannot close them. — bardulia, Sep 04 '13 at 09:15
Why does your algorithm require that, if you don't mind me asking? — user694733, Sep 04 '13 at 09:20
It is a very complicated algorithm, and I cannot change it (more than 6000 code lines). — bardulia, Sep 04 '13 at 09:22
@user694733 I once ran into my limit of 1024, when I used `mmap` to map a whole bunch of tables into memory. All tables together was too big for the maximum file size. — Bryan Olivier, Sep 04 '13 at 09:24

Paul Evans · Answer 1 · 2013-09-04T09:08:28.500

2

If you issue the bash shell command:

ulimit -n

you'll see that 60 is your limit for open file handles. You can change it with:

ulimit -n 256

Note: there're soft (-S) and hard (-H) limits you can see with -Sn and -Hn, you can change your soft limit up to your hard limit.

edited Sep 04 '13 at 09:08

answered Sep 04 '13 at 09:02

Paul Evans

27,315
3
37
54

That's my guess as well, but 60 (64 probably) is quite low as open files limit. For example my Linux box has 1024. – cnicutar Sep 04 '13 at 09:04
256 for my Mac, but certainly plausible it is lower on his rig. – WhozCraig Sep 04 '13 at 09:04
I forgot to say that I am in Windows. How can I change it in there? – bardulia Sep 04 '13 at 09:12

Tom Tanner · Answer 2 · 2013-09-04T12:57:51.953

2

There's actually two things that constrain how many files you can have open at any time:

The environment limit specified by ulimit -n.
The C runtime library. I know of several that limit you to 256 file handles (Sun to name one)

Your current limit is probably 63 once you take into account STDIN, STDOUT and STDERR already being opened, and I don't know of a system that goes that low so it's probably your ulimit but you need to be aware of the other limit.

On windows you can use _setmaxstdio(n) but in the default case you should still be able to open 512 files. so I'm still a little confused as to why you only get 60 odd unless you open each file about 8 times...

edited Sep 04 '13 at 12:57

answered Sep 04 '13 at 09:17

Tom Tanner

9,244
3
33
61

Tom, my C runtime library allows 1024 files, so it must be the environment limit. About the environment limit, is it enough with ulimit -100 statement? – bardulia Sep 04 '13 at 09:20
I don't think ulimit applies to windows – Tom Tanner Sep 04 '13 at 09:23
Sorry, I meant _setmaxstdio(n). I will try it. – bardulia Sep 04 '13 at 09:31

fopen does not deal with more than 60 files at the same time

2 Answers2

Linked