0

During some process, I'm storing objects to a file (serialization). Each batch will store 100 files. I'm quickly getting "Too many open files" error.

I'm closing the StreamWriter immediately after usage and I also tried to dispose the file. I'm having a multithread application, but I centralised the storage into a single thread to make sure that there are no more then X operations in parallel. I'm now even storing object by object (one at a time) and i'm still getting this error.

It looks like "closing" or "disposing" the StreamWriter is not enough. Any idea someone?

I tried a lot of different code, but this is my current code:

            Log.i("Streamwriter opened");
            StreamWriter outfile = new StreamWriter (filePath);
            outfile.Write (content);
            outfile.Close ();
            outfile.Dispose ();
            Log.i("Streamwriter closed");

I printed it also out to make sure that "opening" a file is always immediatly followed by "closing" a file.

Any ideas?

Matt
  • 1,111
  • 11
  • 20
  • I'd guess that the GC is not able to keep up with the number of files you're creating. You might try switching to SGen to see if that helps - http://www.mono-project.com/Working_With_SGen – Jason Mar 06 '14 at 16:48
  • 2
    My guess is that there are other open files. You should be able to use Instruments and one of the file-related templates to view disk activity and see which file is being kept open. – Rolf Bjarne Kvinge Mar 06 '14 at 18:17
  • Switching GC, as suggested by @Jason, won't help - that's why using `IDisposable` is the *right* thing to do. Something else is likely using too many files (like @Rolf said). See http://stackoverflow.com/q/15886034/220643 and links for a bit more information. – poupou Mar 06 '14 at 19:17
  • Even if you're centralizing it: how do you prevent that 100 calls to your logging happen at the same time? Is there a `lock()` somewhere in your code? – Krumelur Mar 06 '14 at 19:39
  • Any reason you're not using, well, using: "using (var ourfile = new StreamWriter(filePath) {}"? – SKall Mar 07 '14 at 01:44
  • @Skall: I was using a "using" block. Just out of desperation, I tried to manually dispose it. – Matt Mar 08 '14 at 16:02
  • @Krumelur: there is indeed a "lock" in my code (before calling this function to save data to the HD) – Matt Mar 08 '14 at 16:03

2 Answers2

1

I already tried Instruments and I/O templates before posting this question, but couldn't find anything strange there. I now tried some other methods from following link: on iOS/iPhone: "Too many open files": need to list open files (like lsof) (the link was posted in the stackoverflow thread posted by poupou in his comment)

The best method for me was just to type in "lsof" in Terminal. I immediately saw that there were a lot of open files of my database. Before serialisation, I also write the ID of the object to my database. Because the error "Too many open files" always pointed to my serialisation code (and not my database code), I was looking there for a problem.

When inserting an object into my database, I'm using "SqliteConnection". I was correctly closing AND disposing this object, however, it seems this is not enough to really also close my database file.

This URL was helpful to me: SQLite keeps the database locked even after the connection is closed

I indeed did not dispose the "SqliteCommand" itself.

After fixing this, I do not receive this error anymore.

Thanks!

    SqliteConnection cnn = new SqliteConnection(dbConnection);
    cnn.Open();
    SqliteCommand mycommand = new SqliteCommand(cnn);
    mycommand.CommandText = sql;
    object value = mycommand.ExecuteScalar();
    mycommand.Dispose ();     // <= THIS FIXED THE ERROR
    cnn.Close();
    cnn.Dispose ();
Community
  • 1
  • 1
Matt
  • 1,111
  • 11
  • 20
  • 1
    Yes, that makes more sense than the StreamWriter. Also a good demonstration why 'using' is preferred when you need to dispose the object on the same scope. Makes it easier to read (in my opinion) as well. – SKall Mar 08 '14 at 18:05
  • Thanks a lot, I hope that will fix that strange error in my application, too. As SKall said, you should always use an using block around disposable objects: http://www.dotnetperls.com/using (I can't actually in my special case, because I am collection SqliteCommands within a unit of work session, so I am now disposing them after committing the session). – asp_net Apr 10 '14 at 09:38
0

If you are using FileSystemWatcher it can be the reason of the problem: SO post

When we disabled FileSystemWatcher in our project the error disappeared (we experienced the same error for Xamarin.Mac though, but probably it's relevant for Xamarin.iOS too).

DenisL
  • 308
  • 2
  • 10