6

I am making an app and it needs to open at least 676 files simultaneously to a maximum of 1400 files. I will be writing to these files using the StreamWriter Class and reading the data using StreamReader Class. So, is there a maximum limit on the no of files that can be opened simultaneously for reading or writing in C# just like VC++ as described in the following link. Is there a limit on number of open files in Windows.

Community
  • 1
  • 1
Pratik Singhal
  • 6,283
  • 10
  • 55
  • 97
  • 4
    Are you hitting the limit or are you just asking before trying anything? – Austin Salonen Nov 29 '13 at 16:06
  • 3
    Can you explain why you need to do this? There's nothing wrong with your question, just for my own curiosity! – JMK Nov 29 '13 at 16:06
  • 1
    So why not just try to open 1400 files simultaneously and find out for yourself? – Randy Minder Nov 29 '13 at 16:08
  • @JMK Currently , I am performing the same "task" with about 400 files open, but I cam perform this "task" better when I create more data ie (write more data and read more data).So, before changing the implementation of the algorithm required to do that task, I just wanted to evaluate whether it is feasible to write and read from so many files at the same time. – Pratik Singhal Nov 29 '13 at 16:09
  • 2
    What is the task I think JMK is getting at, I'm also interested. –  Nov 29 '13 at 16:10
  • 1
    @Voidpaw I am not talking about `OpenFileDialog`, I am talking about "Reading" and "Writing" to the file using `StreamReader` and `StreamWriter` respectively. – Pratik Singhal Nov 29 '13 at 16:11
  • @Pratik My bad, I'll remove the flag. – Voidpaw Nov 29 '13 at 16:12
  • @DeeMac, this is not the exact task, but an analogy to what the task is: Suppose, I give you a large set of random strings say about 100 million and ask you(sometime later but not as soon as I give you the strings) to find all the strings which contain a particular substring. Then, to make the task faster, as soon as the user gives us those random string we can classify those strings in some way or the other and write to a file for future references, and when in the future we have a search query then we can just choose the desired file and search its contents and present the result to the user – Pratik Singhal Nov 29 '13 at 16:28
  • @DeeMac continuation to previous comment, the classification depends upon the task given, by choosing a suitable classification, we can make the search operation faster and faster, as we do a bigger classification, the more files we need to write and read. – Pratik Singhal Nov 29 '13 at 16:30
  • You can open an absurd number of files without trouble. Why you'd want to is another question entirely. It's likely that the classification you're talking about can be done in a more reasonable way. But if you want help with that, you'd post a different question. – Jim Mischel Nov 29 '13 at 16:45
  • @JimMischel I am entirely satisfied with my classification. The only doubt which I had was clarified by your comment. – Pratik Singhal Nov 29 '13 at 16:47
  • http://blogs.technet.com/b/markrussinovich/archive/2009/09/29/3283844.aspx – Hans Passant Nov 29 '13 at 17:04
  • @Pratik It sounds like you may want to use a database instead of flat files. – Daniel Mann Nov 29 '13 at 17:38
  • @DanielMann - yep, I concur. It sounds to me Pratik that you've got the exact kind of problem databases have been designed for. Storing the data is primitive to a DB (tables), searching them too (SQL with WHERE clauses). And when you imminently come across performance issues, you'll attempt to refactor code, but could've just used indexes, for example. Further to all of what I've said above, I might be wrong - but using an analogy to your problem without giving specifics won't help me comment any more accurately. –  Nov 30 '13 at 19:32
  • @DainelMann, Deemac , Thanks for the suggestions, I will surely look into my problem and how could databases simplify it. – Pratik Singhal Dec 01 '13 at 04:19

2 Answers2

12

The upper limit on files opened by .NET is governed by the limit imposed on the Win32 API CreateFile, which is 16384.

bhouston
  • 955
  • 9
  • 14
7

This works for me:

  var streams = new Stream[10000];
  for (var i = 0; i < streams.Length; i++) {
    streams[i] = File.OpenWrite(Path.Combine(Path.GetTempFileName()));
    streams[i].WriteByte((byte)'A');
  }
  var tasks = new Task[streams.Length];
  for (var i = 0; i < streams.Length; i++) {
    var index = i;
    tasks[i] = new Task(() => {
      streams[index].WriteByte((byte)'B');
    });
    tasks[i].Start();
  }
  Task.WaitAll(tasks);
  for (var i = 0; i < streams.Length; i++) {
    streams[i].Close();
  }
Ondrej Svejdar
  • 21,349
  • 5
  • 54
  • 89