1

I know that number of items you can add to List in C# is limited by int (because that is the type of count). Which should be around 2 billion.

But my question is say I want to have such list:

List<MyClass>= new List<MyClass>;

Where 

class MyClass
{
  string s1; // should be string of 8 characters always
  DateTime t;
}

How many objects of MyClass I can add to the list? I assume now I have to take into account that the memory it will occupy is numberOfElements * sizeof(MyClass)? So how many elements I can add using such constraint? (I believe I may run out of memory faster than I reach max value of int due to formula above, isn't it?).

  • 1
    Since `MyClass` is a reference type, the list will just contain references to objects, so the size of the list will not be any different. – D Stanley Mar 30 '15 at 13:54
  • You can add as many elements as memory will allow up to an upper limit of 2GB per object pre .NET 4.5 OR up to to the maximum size of int. – Ian P Mar 30 '15 at 13:54
  • the number of object is limited by the int upper bound limit. The limitation of memory depends on if you are running 32 or 64 bits. Also `SizeOf(class)` wont give you the actual size the class is taking into memory. it's a good estimate but not accurate – Franck Mar 30 '15 at 13:55
  • not the same question, but checkout the answer on this topic: http://stackoverflow.com/questions/7885294/list-size-limitation-in-c-sharp – Caio César S. Leonardi Mar 30 '15 at 13:55
  • It wouldn't be hard to write a test application to test it out on a similar machine to the machine that will be in production. If you are running in IIS, .net version, etc... – Nick Bray Mar 30 '15 at 13:57
  • @Franck: So *number* of elements is still bound by int right? But how much memory does MyClass take here in your opinion? And how many such elements will I be able to add say on a 4GB RAM machine? 32 bit –  Mar 30 '15 at 14:00
  • @IanP: I could not follow you. Can you please help me estimate? How much virtual memory is typically available say I have 4GB RAM, 32 bit machine. How many elements can I add of type `MyClass`? –  Mar 30 '15 at 14:06
  • Is it definitely a list that you need? There are other datatypes / data storage types available that might have a better use.. – Sayse Mar 30 '15 at 14:06
  • @Sayse: I need to often query large database for result I thought loading it to list and querying list will be faster then performing SELECT each time on database, I was wrong? –  Mar 30 '15 at 14:21
  • @user20034 - Hard to tell, you'd be better off profiling it, I would imagine you'd be better off doing the query on the database though since that significantly reduces the overhead of returning data – Sayse Mar 30 '15 at 14:29
  • @Says: Yes with database I will just get yes or no result if record which I am looking for is there - but that way I need to query database each time? This way I need to load all database to list and query list. –  Mar 30 '15 at 14:31
  • Why do you? you can select elements that match your query and then return them instead. – Sayse Mar 30 '15 at 14:36
  • @Sayse: No I need to check if some record is in database. So I just do SELECT. If I get NULL record is not in database. My point was I may need to query the database for each record I need to check separately- and there are many records to check. So I thought loading database to list and querying list was faster –  Mar 30 '15 at 14:38
  • Again, hard to tell, but you can always profile a query that uses `EXISTS` – Sayse Mar 30 '15 at 14:40
  • @user20034 Yes it is still limited by integer maximum value. Getting the actual class memory usage is not hard but not quite easy. You can hook to the process info and note the increase of memory used but the easiest way to do it is by using ANTS memory profiler. this tool give you the exact amount your object is currently using. – Franck Mar 30 '15 at 17:37
  • @Franck: what about object max size being 2gb ?http://stackoverflow.com/questions/3906891/what-is-the-max-limit-of-data-into-liststring-in-c –  Mar 30 '15 at 20:39
  • @user20034 Yes for a SINGLE object the limit was around 2gb 4-5 years ago. Don't know if that changed since then. I never worked with an object lager than 400-500 mb. What i meant is by using `SizeOf` being not accurate, i meant that on an object it might return 200kb while in reality it actually take 260kb. ANTS memory profiler give you much more accurate number. Last time i had to use it was quite accurate. I would say max 5% off. – Franck Mar 31 '15 at 13:40

1 Answers1

5

How many objects of MyClass I can add to the list? I assume now I have to take into account that the memory it will occupy is numberOfElements * sizeof(MyClass)?

Since MyClass is a reference type, the list itself will just contain references to objects, so the maximum size of the list will not be any different.

I believe I may run out of memory faster than I reach max value of int due to formula above, isn't it?

You may run out of memory, yes, but it's not a limitation of List<T>. You are only limited by the amount of virtual memory available.

D Stanley
  • 149,601
  • 11
  • 178
  • 240
  • @D Stanley: Ok so*number* of elements is still limited by size of int right? But how many elements can I add considering they still take space somewhere in memory? –  Mar 30 '15 at 13:58
  • @user20034 It depends on how much virtual memory you have available and how big the strings are (since they'll take up virtual memory too). – D Stanley Mar 30 '15 at 13:59
  • @D Stanley: String will be 8 characters long. I need to estimate. Say 4GB RAM 32 bit machine? How big is virtual memory typically on such machine say? –  Mar 30 '15 at 14:01
  • @user20034 Calculate the space take by the strings (8*2*n bytes) since strings in c# are 16-bit unicode), the string references (8*n), the DateTimes(8*n bytes), the list references (8*n), and solve for n given the amount of _virtual_ memory you have (which should be more than 4GB). This will be a _very_ rough estimate and will probably be way too high. I would say halve the number to come up with a practical maximum. – D Stanley Mar 30 '15 at 14:06
  • @user20034 Or create objects and add them to the list until you get an `OutOfMemoryException`. Also note that virtual memory is shared by all running apps (and the OS) on the machine, which will rise and fall over time, so it's a moving target. – D Stanley Mar 30 '15 at 14:09
  • @D Stanley: Should not it be: (1) String objects, which is 8 * 2 * N, (2) String references 8 * N (3) Date Time references, 8 * N and (4) Date Time objects 8 * N, and (5) and also list references 8 * N ? (what do you mean by list references?) –  Mar 30 '15 at 14:10
  • `DateTime` is a struct, not a reference type, so it will be stored alongside the string reference. The list is essentially an array of references to objects of your class. Each reference is 8 bytes on a 32-bit machine. – D Stanley Mar 30 '15 at 14:12
  • @D Stanley: Using your formula and 4 GB virtual memory I received: 107374182 am I correct? So I can count to create 107374182 objects of type MyClass and add them to list? –  Mar 30 '15 at 14:12
  • @user20034 4GB of _physical_ memory or _virtual_ memory? (and I don't mean virtual in the sense of the specs of a VM). Virtual memory is used by other running apps and the OS so it's not absolutely determinable. It will change over time so you need to look at the machine in steady state to get a baseline and than calculate a _theoretical_ maximum. – D Stanley Mar 30 '15 at 14:15
  • @D Stanley: I wanted a rough estimate ... ? Did I at least get this number right: 107374182. I can't even assume 2 GB virtual memory on such machine? –  Mar 30 '15 at 14:17
  • @user20034 Your arithmetic is right _assuming_ 4GB of virtual memory, It may be more or less depending on what else is running on the machine. – D Stanley Mar 30 '15 at 14:23
  • @D Stanley: Thanks so I can count on creating 107374182 objects and add to list if 4GB is available. But if I have 4GB RAM 32 bit machine, can I safely assume I will at least have 2GB virtual memory? –  Mar 30 '15 at 14:25
  • @user20034 "If 4GB is availble" probably, but it's not guaranteed. you could also have some other app running at the same time that is using up memory. Virtual memory is not a constant, it changes over time and you have little (if any) control over it. – D Stanley Mar 30 '15 at 14:30
  • @D Stanley: Thanks and finally maybe you can give advice. I either have to load whole database to list and query list (then get maybe this memory problems) or just query the database? Which is better? Just for querying database if some record is there I need to do it very often. That is why I thought loading it to list and querying list would be faster –  Mar 30 '15 at 14:32
  • @user20034 It would have been helpful to explain your _actual_ question rather then asking for a theoretical limit that is not determinable. Ask your _real_ question as a new question - there are many more variables than just memory usage. – D Stanley Mar 30 '15 at 14:35
  • @D Stanley: it will be probably closed because of opinion based-you didn't get the scenario fully? Please also see my comments with Sayse above –  Mar 30 '15 at 14:39
  • @D Stanley: What about object max size being 2gb http://stackoverflow.com/questions/3906891/what-is-the-max-limit-of-data-into-liststring-in-c ? –  Mar 30 '15 at 20:36
  • @user20034 Right, and a list of 100 million references will be about 400 million bytes, well under the 2GB maximum. The size of the objects being referenced does not count towards the 2GB limit. I really think you're asking the wrong question - you're asking _can_ I have a list of 100 million records when you _really_ need to ask if you _should_. – D Stanley Mar 30 '15 at 21:58