0

In WPF, I want to effectively store the millions of objects in low memory usage and retrieve it very fast. Below is my sample class.

public class CellInfo

{

public int A { get; set; }

public int B { get; set; }

public string C { get; set; }

public object D { get; set; }

public bool E { get; set; }

public double F { get; set; }

public ClassA G { get; set; }

}

I want to store a millions of CellInfo objects and each object have its own identity. And I want retrieve it back using that identity. If the properties of the CellInfo instance is not defined, then it needs to be return the default value which would be stored in a static field.

So i want to only store the Properties of CellInfo object which are defined and others i dont want to keep in a memory and can retrieve those from static variable.

So can anyone please suggest me fastest way to store and retrieve the millions of objects in a low memory usage?

Note: I dont want any additional software installation and DB or any external file to store this.

Selvamz
  • 362
  • 3
  • 16
  • 1
    How will you be retrieving them? one at a time sequentially? one at a time randomly? several at a time? – Amleth Oct 21 '14 at 05:12
  • I want to retrieve one at a time randomly. For eg. If i stored it in a dictionary, var cell = Cells[id]. But the problem it takes huge memory when i stored millions of CellInfo object. My CellInfo class have around 25 Properties.... But I would change only a few properties (2 or 3) of CellInfo instance and others needs to return the default value. So i want to keep/store only the properties which i was defined. – Selvamz Oct 21 '14 at 05:29
  • Why would you not want to use a database for this? If you don't want to use it, you will end up storing all your data in your RAM afaik. You can then use any collection you want, they will probably all take about the same amount of space. – Dion V. Oct 21 '14 at 05:55
  • see if [Improve performance for huge ListBox in StackPanel](http://stackoverflow.com/questions/25530354/improve-performance-for-huge-listbox-in-stackpanel/25530497#25530497) helps. – pushpraj Oct 21 '14 at 07:14

1 Answers1

1

You haven't indicated which field is the 'own identity', so I've assumed a Guid Identity. A Dictionary keyed on this identity should offer fastest retrieval. 

Dictionary<Guid, CellInfo> cells = new Dictionary<Guid, CellInfo>();

If you have the data already, you can use .ToDictionary() to project the key / value mappings from an enumerable.

If you need to simultaneously mutate and access the collection from multiple threads (or if you intend making the collection static), you can swap out with a ConcurrentDictionary to address thread safety issues:

Before accessing an element, you'll need to determine whether the item exists in the Dictionary via ContainsKey (or TryGet as per Amleth). If not, use your default element. So would suggest you hide the underlying dictionary implementation and force consumers through an encapsulation helper which does this check for you.

StuartLC
  • 104,537
  • 17
  • 209
  • 285
  • 1
    I think a TryGet, is better to use then a ContainsKey and then a Get. I guess it depends on whether or not there are more 'default' objects than stored objects. – Amleth Oct 21 '14 at 05:14
  • Yes, and your point about 'how the data will be accessed' is also well made - I'm assuming access by key here. Random queries on other fields will not be performant. – StuartLC Oct 21 '14 at 05:18
  • Thanks for your quick suggestion StuartLC. But I already tried using Dictionary. But the problem it takes huge memory when i stored millions of CellInfo object instances into a Dictionary. My CellInfo class have around 25 Properties.... But I would change only a few properties (2 or 3) of CellInfo instance and others needs to return the default value. So i want to keep/store only the properties which i was defined. – Selvamz Oct 21 '14 at 05:36
  • The [Sparse Matrix](http://en.wikipedia.org/wiki/Sparse_matrix) pattern might be applicable here, although this is typically for simple data types, not graphs (assuming `ClassA` can be complex itself). From a memory perspective, the main things to worry about are the heap fields - `object`, `string` and `ClassA`. Strings are immutable, so you can assign the reference to the default. If `ClassA` is pure (can't be changed, even by setters), you could also assign a default cell's ClassA to the default, although will need to ensure immutability to prevent the default reference becoming corrupted. – StuartLC Oct 21 '14 at 05:48