0

I have a list of String like this

   private object = new object();
    public List<string> list {
     get{
           lock(object){
            return _list;
           }
       }
    }
    List<string> _list = new List<string>();
    //some event which trigger's 1000 in a second
    public void  event(object sender, string e){
              list.add(e);
    }
    public void processList(){
       //process the list items one by one  and remove it.      

    }
    public void writeOnDisk(){
             //write list on Disk
    }

now my problem is List Processing Rate is slower than populate. And I want to take the backup of the list on disk for application crash or system failure but afraid if write list Disk than it will slow down the performance. I do not want to loose the data even a single item. Can you suggest me best approach in this regard

sm.abdullah
  • 1,777
  • 1
  • 17
  • 34
  • I recommend changing your title to one which summarizes your problem. Your title could be used for tens of thousands of questions on this site. – John Saunders May 26 '15 at 09:58
  • Performance is relative. What exactly are you trying to achieve? From your description it sounds like you are trying to implement some sort of job queue? Please be a little clearer regarding "slow", "imrpove" and of course what exactly "processing" means. – LocEngineer May 26 '15 at 09:59
  • If you need to persist the list, you will lose some performance while persisting it (or at least you will using your current implementation). Whether that performance loss (which can be _very_ small) is acceptable, is up to you. We can't answer this. If you may not lose a single item, you may want to consider a different approach altogether. – CodeCaster May 26 '15 at 09:59
  • isnt it should be _list.add(e); ?? – Neel May 26 '15 at 10:00
  • @Neel, no. OP created a locking wrapper for `_list`, called `list`. – CodeCaster May 26 '15 at 10:01
  • @LocEngineer yes i want So called JOB Queue. actually job rate is 1000/sec in worst case and job completion rate is 150/sec – sm.abdullah May 26 '15 at 10:15
  • @CodeCaster actually i do not want to loose a single item in the list. If i take backup after a second I think due to locking it will cause huge performance issue. – sm.abdullah May 26 '15 at 10:15
  • 1
    Don't think, measure. – CodeCaster May 26 '15 at 10:15
  • 2
    The `lock` as written is totally useless. It is released after the `return` and before the `Add` – xanatos May 26 '15 at 10:16
  • @xanatos what is the way write lock to make all list operation thread safe in a single step instead of rewriting each function i.e by overriding ,overloading. – sm.abdullah May 26 '15 at 10:24

1 Answers1

0

Use Thread Safe Collections for .Net 4.0 or above. For your case, consider using ConcurrentQueue<T>

Thread safe collections in .NET

And one of the bottleneck should be the I/O of writing files, you may consider to write by batch instead of line by line.

Community
  • 1
  • 1
Eric
  • 5,675
  • 16
  • 24
  • What question are you answering? – CodeCaster May 26 '15 at 10:02
  • @CodeCaster OP is asking the performance hit using the list, it is actually the `lock` degrades the performance. Using `ConcurrentQueue` can release the expensive waiting kernel transition. – Eric May 26 '15 at 10:08
  • You may want to mention that in your answer. OP is also asking something like _"how to continuously persist a list?"_, so if using a different type of collection doesn't solve (much) of the problem _"items are being added faster than they are being processed"_, then this is only half an answer. – CodeCaster May 26 '15 at 10:10
  • i think concurrent Collection are not efficient like list i mean queue in some cases. – sm.abdullah May 26 '15 at 10:18
  • The concurrent collections are not necessarily a good choice for all scenarios, even when collections are accessed from multiple threads. In many cases a simple locking scheme can still be faster. – Dirk May 26 '15 at 10:27