0

I want to create a list, that will handle A LOT (number of items will be stored in ulong) of items inside. I know, its a bit too much (int is sufficient for this task), but I was just wondering is this possible. Mainly: normal List can take as argument for operator [] only int. I suppose its a limit of Lists. How to make larger Lists? Is there any other class, or would I have to work with List<List<byte>>?

SkillGG
  • 647
  • 4
  • 18
  • The term 'a lot' is a bit broad. Is there any change to elaborate this with real values? – Dimitar Apr 05 '19 at 05:21
  • It will be a bit more than `UInt32` (`uint`) (4,000,000,000+) can handle. – SkillGG Apr 05 '19 at 05:21
  • 3
    If your list is really that long, you should probably use a file or a database. Putting that thing in memory is not going to end well. – John Wu Apr 05 '19 at 05:24
  • I was just asking how to do such list. I know, that somewhere arond 4,000,000 objects stored would be fatal, but is there a way to create list that would handle such big numbers? (Btw. I cannot create a file/database. I will just wisely delete unused data from the list) – SkillGG Apr 05 '19 at 05:25
  • You cannot put for indexer another type except `int`. – Dimitar Apr 05 '19 at 05:26
  • 2
    your list will have more then `2147483647` items? if each item will ocupy only 24 bytes (minimal object size for Microsoft .NET CLR v4 on x64) this will be about 48GB of memory. Is this is not enough? – vasily.sib Apr 05 '19 at 05:27
  • Possible duplicate of [what is the max limit of data into list in c#?](https://stackoverflow.com/questions/3906891/what-is-the-max-limit-of-data-into-liststring-in-c) – Martin Verjans Apr 05 '19 at 05:29
  • You can use indexers overload to add your type of indexer. – Dmytro Mukalov Apr 05 '19 at 05:31
  • 1
    I suppose you could write your own class that uses a 2D array as a backing store. The array could theoretically store `int32` x `int32` items. Another option is to use a MemoryStream for storage and use Seek() to index into it. – John Wu Apr 05 '19 at 05:31
  • @JohnWu I thought about it, but haven't anyone done this yet? I suppose it's going to be my task for today :) – SkillGG Apr 05 '19 at 05:32
  • @MartinVerjans it is not a duplicate for that. I was wondering how to do larger list, not what is the biggest possible size. – SkillGG Apr 05 '19 at 05:33
  • Either way storing that sort of data in the LOB is going to create problems one way or another sooner or later especially if you are letting this auto expand – TheGeneral Apr 05 '19 at 05:34
  • 3
    Am I the only one who haven't got more than 48Gb of RAM to store such huge lists? – vasily.sib Apr 05 '19 at 05:35
  • @vasily.sib what about list of bytes? It would not require 24B for one object will it? `List>` – SkillGG Apr 05 '19 at 05:36
  • 1
    @SkillGG, sure not (as `List` is holding items in `T[]` internally, so `List` is actually a `byte[]`), 24B is a minimal **object** size. But for 4,000,000,000+ `byte` items this list will be still 4+Gb of RAM. – vasily.sib Apr 05 '19 at 06:13
  • 1
    @SkillGG, also, as you working with this huge amount of data, your program can't run on `x86` so don't forget to switch your target from `AnyCPU` to `x64` in your Visual Studio – vasily.sib Apr 05 '19 at 06:17

1 Answers1

0

I would go with a Dictionary<ulong, T>.

As long as the size of your Dictionary doesn't exceed the maximum authorized size, you will be fine.

Martin Verjans
  • 4,675
  • 1
  • 21
  • 48
  • Yes! That's what I was thinking about. I knew, that there has to be a way. That I don't have to do any `List>` shenanigans or (worse) make own class for it. – SkillGG Apr 05 '19 at 14:15