Using Long Integer types with collection methods

Question

I've encountered a problem, which is best illustrated with this code segment:

public static void Foo(long RemoveLocation) 
{
    // Code body here...

    // MyList is a List type collection object.
    MyList.RemoveAt(RemoveLocation);
}

Problem: RemoveLocation is a long. The RemoveAt method takes only int types. How do I get around this problem?

Solutions I'd prefer to avoid (because it's crunch time on the project):

Splitting MyList into two or more lists; that would require rewriting a lot of code.
Using int instead of long.

So is it 4 billion, 9 billion, 10 trillion? The space between `int.MaxValue` and `long.MaxValue` is a *lot*, and potentially could affect answers. — Servy, Nov 08 '12 at 19:29
The .NET `List` class only supports up to an `int`'s worth of items... http://stackoverflow.com/questions/3906891/what-is-the-max-limit-of-data-into-liststring-in-c — Tyler Lee, Nov 08 '12 at 19:33
Err. Is this an XY problem? What problem are you trying to solve that requires such a massive list in the first place? — Bradley Thomas, Nov 08 '12 at 19:43
You shouldn't even have lists of 1/100th of that size, it simply gets highly inefficient, especially if you modify them. — Martin Baulig, Nov 08 '12 at 19:49
@BradThomas Yes, it's a very large set of X,Y points. Very, very large sometimes. — kevin628, Nov 08 '12 at 20:44

Martin Baulig · Answer 1 · 2012-11-09T02:49:57.170

In theory, the maximum number of elements in a list is int.MaxValue, which is about 2 billion.

However, it is very inefficient to use the list type to store an extremely large number of elements. It simply has not been designed for that and you're doing way better with a tree-like data structure.

For instance, if you look at Mono's implementation of the list types, you'll see that they're using a single array to hold the elements and I assume .NET's version does the same. Since the maximum size of an element in .NET is 2 GB, the actual maximum number of elements is 2 billion divided by the element size. So, for instance a list of strings on a 64-bit machine could hold at most about 268 million elements.

When using the mutable (non-readonly) list types, this array needs to be re-allocated to a larger size (usually using twice the old size) when adding items, requiring the entire contents to be copied. This is very inefficient.

In addition to this, having too large objects could also have negative impacts on the garbage collector.

Update

If you really need a very large list, you could simply write your own data type, for instance using an array or large arrays as internal storage.

There are also some useful comments about this here: http://blogs.msdn.com/b/joshwil/archive/2005/08/10/450202.aspx

score 0 · Answer 2 · answered Nov 08 '12 at 21:05

If there was a way you could group similar items together, could you bring the total down below the limit? E.g. if your data contains lots of repeated X,Y coords, you might be able to reduce the number of elements and still keep one list, by creating a frequency count field. e.g. (x,y,count)

Using Long Integer types with collection methods

2 Answers2