2

I want my cake and to eat it.

I like the way Lists in C# dynamically expand when you go beyond the initial capacity of the array. However this is not enough. I want to be able to do something like this:

int[] n = new int[];    // Note how I'm NOT defining how big the array is.
n[5] = 9

Yes, there'll be some sacrifice in speed, because behind the scenes, .NET would need to check to see if the default capacity has been exceeded. If it has, then it could expand the array by 5x or so.

Unfortunately with Lists, you're not really meant to set an arbitrary element, and although it is possible if you do this, it still isn't possible to set say, the fifth element straight away without initially setting the size of the List, let alone have it expand dynamically when trying.

For any solution, I'd like to be able to keep the simple square bracket syntax (rather than using a relatively verbose-looking method call), and have it relatively fast (preferably almost as fast as standard arrays) when it's not expanding the array.

Community
  • 1
  • 1
Dan W
  • 3,520
  • 7
  • 42
  • 69
  • 2
    What would you expect the previous 4 elements to contain in your code example? – Zohar Peled May 20 '15 at 14:23
  • 3
    Arrays have a fix size so you want a list. Create your own list that inherits from `List` and add the functionality you need, f.e. change the [indexer](https://msdn.microsoft.com/en-us/library/6x16t2tx.aspx) accordingly so that it increases the size of the list if needed. – Tim Schmelter May 20 '15 at 14:24
  • You can too set arbitrary elements by index in Lists... you just have to add the space manually first. – Joel Coehoorn May 20 '15 at 14:24
  • @ZoharPeled: Anything at all - ints, strings, doubles, even generic objects - you name it. – Dan W May 20 '15 at 14:24
  • You can write your own implementation to do this. – Matthew May 20 '15 at 14:26
  • 4
    So you want a `Dictionary`? You could inherit dictionary, like `public class MyCoolType : Dictionary { }` which would then let you define something like `MyCoolType n = new MyCoolType()` then you can do exactly what you want, `n[5] = 9;` – Ron Beyer May 20 '15 at 14:26
  • 1
    @DanW Sorry, but this doesn't make any sense whatsoever. How do you populate an int array with strings or other non-int values? Note Jon Skeet's asnwer to the post you linked to, I think this should probably be the way to go. – Zohar Peled May 20 '15 at 14:30
  • 1
    @ZoharPeled: I meant the type could be `string[]`, `double[]`, `int[]` etc. in the first place. Okay, change my initial response to just 'ints'. – Dan W May 20 '15 at 14:32
  • 1
    Dan W, Are your indexes sparse or dense? For sparse @RonBeyer's suggestion to use dictionary is likely better. – Alexei Levenkov May 20 '15 at 14:35
  • @DanW Ok, but what values? for reference type you can just use null for the first 5 places in the array, but for primitive types such as int, double and so on you can't use null. so what value would you like to use? **note** that the default value of int is 0, but it might not be a good option for you since it is a valid value in most scenarios. – Zohar Peled May 20 '15 at 14:37
  • @RonBeyer: That looks like an interesting solution. However, I fear it would be pretty slow for general use. – Dan W May 20 '15 at 14:42
  • 1
    @DanW, why? Hash table look ups are O(n), dictionaries are optimized to be extremely fast. Its not as fast as an array, granted, but especially if you change to a `SortedDictionary` it should be comparable to the `List` implementations. – Ron Beyer May 20 '15 at 14:43
  • @RonBeyer: By all means, add it as an answer. I'll check it out, and may perform some speed tests on it. – Dan W May 20 '15 at 14:44
  • @RonBeyer I think you mean hash table look ups are O(1) not O(n) – juharr May 20 '15 at 14:45
  • @ZoharPeled: Oh I see. Either 0 for ints/doubles, or if we're creating a string array, empty ""s or nulls are fine. – Dan W May 20 '15 at 14:47
  • @juharr, yes, sorry, I was thinking List lookups, I have that in my answer. – Ron Beyer May 20 '15 at 14:49
  • @RonBeyer SortedDictionary is slower (O(logN)) for all operations, I'm not sure why it would be better that regular Dictionary when implementing sparse array.. – Alexei Levenkov May 20 '15 at 14:50
  • @AlexeiLevenkov yes, you are right, I wasn't thinking straight. See my answer, I think I have it right there. – Ron Beyer May 20 '15 at 14:52

4 Answers4

6

Note that I don't necessarily advocate inheriting List, but if you really want this:

public class MyList<T> : List<T>
{
    public T this[int i]
    {
        get { 
            while (i >= this.Count) this.Add(default(T));
            return base[i]; 
        }
        set { 
            while (i >= this.Count) this.Add(default(T));
            base[i] = value; 
        }
    }
}

I'll add that if you expect most of the values of your "array" to remain empty over the life of your program, you'll get much greater efficiency by using a Dictionary<int, T>, especially as the size of the collection grows large.

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794
  • 2
    Regarding your first sentence, for reference: [Why not inherit from `List`?](http://stackoverflow.com/questions/21692193/why-not-inherit-from-listt) – James Thorpe May 20 '15 at 14:29
  • 2
    @JamesThorpe this is actually one of few cases when inheriting from list makes sense. You are extending collection functionality. – Andrey May 20 '15 at 14:32
  • 1
    Yes, this is a perfect example of a mechanism that extends the List mechanism. – Tim Schmelter May 20 '15 at 14:33
  • @Andrey I agree in this instance - just wanted to use that as an example expanding on the point Joel was making. – James Thorpe May 20 '15 at 14:33
  • 3
    `List` doesn't have a `Length` property. I think you want `Count`. – juharr May 20 '15 at 14:34
  • I don't get it right. He just wants to access elements by his index and still wants to use List ? List does actually have [index property](https://msdn.microsoft.com/en-us/library/0ebtbkkc(v=vs.110).aspx)?! – ckruczek May 20 '15 at 14:37
  • @ckruczek The default implementation will throw an `ArgumentOutOfRangeException` for items outside the current range though, rather than filling the "missing" entries with default values. – James Thorpe May 20 '15 at 14:39
  • With `MyList l = new MyList();` and `l[5] = 9;`, I get the error: "Index was out of range" on the `base[i] = value;` bit. – Dan W May 20 '15 at 14:39
  • 2
    @DanW I think it might need to be `i >= this.Count`? – James Thorpe May 20 '15 at 14:41
  • Excellent! This now works as expected. I think I'll run some speed tests for both get and set when I see more answers. – Dan W May 20 '15 at 14:51
  • It'd expect an measurable but insignificant slow down for the case where the element already exists or you just add the end, but perhaps more significant in situations where your really are using a sparse array... and again, in that situation you should really look at Dictionary – Joel Coehoorn May 20 '15 at 15:00
  • A small but noticeable issue here is with densely packed arrays, you can't insert indexes higher than the physical memory boundary. Try inserting at `int.MaxValue`, you get an out-of-memory exception. Its the same problem with all densely packed arrays, not just this solution. – Ron Beyer May 20 '15 at 15:08
  • Any easy way of extending this to 2D arrays? I tried `var t = new MyList>(); t[2][5] = 9;` without any luck. – Dan W May 20 '15 at 19:21
  • @DanW The `default(T)` expression for the inner list is `null`. You can extend it to work by checking for that special case and instead of `default(T)` initializing to an empty MyList. – Joel Coehoorn May 20 '15 at 19:28
  • @JoelCoehoorn: Any chance of editing your answer to include the code for the 2D version? If you don't want to add to the answer, [pastebin.com](http://pastebin.com/) is a very easy way to show text. I tried a couple of things to get your suggestion to work, but failed. – Dan W May 20 '15 at 20:54
3

A simple solution to the problem is to inherit from Dictionary<TKey, TValue> and just use the value generic:

public class MyCoolType<T> : Dictionary<int, T> { }

Then you would be able to use it like:

MyCoolType<int> n = new MyCoolType<int>();
n[5] = 9;

And a note on performance.

  • For insertions, this is much faster than a list since it does not require you to resize or insert elements at arbitrary positions in an array. List<T> uses an array as a backing field and when you resize it, it is expensive. (Edit: Lists have a default size and its not always that you are resizing it, but when you do, its expensive)
  • For look-ups, this is very nearly O(1) (source), so comparable to an Array look-up. Lists are O(n), which get progressively slower as you increase the number of contained elements.
  • Sparsely packing is much more memory efficient than using a List with dense packing as it doesn't require you to use empty items just to reach a specific index.

Other Notes:

  • In the other solutions, try inserting an item at index 570442959 for example, you'll get an OutOfMemoryException thrown (under 32 bit, but even 64-bit has problems). With this solution you can use any conceivable index that the int type supports, up to int.MaxValue.
  • Lists don't allow negative indexes, this will.
  • MyCoolType.Count is the equivalent of the array Length property here.

Here are the results of my performance test:

  • Inserting 1 million elements into MyList: 29.4294424 seconds
  • Inserting 1 million elements into CoolType: 0.127499 seconds
  • Looking up 1 million random elements MyList: 1.6330562 seconds
  • Looking up 1 million random elements CoolType: 1.304348 seconds

Full source to tests here: http://pastebin.com/kEdLgFaw

Note, to run these tests I had to set to X64 build, debug, and had to add the following to the app.config file:

<runtime>
  <gcAllowVeryLargeObjects enabled="true" />
</runtime>
Ron Beyer
  • 11,003
  • 1
  • 19
  • 37
  • You really should not be deriving sparse array from dictionary - array is not dictionary to start with... For example "count" is very different between these types. (Side note: adding comment on when to use that approach would make it much better - i.e. something like Joel Coehoorn's remark at the end of the other answer). – Alexei Levenkov May 20 '15 at 14:54
  • This works too! I look forward to trying answers all out, and maybe even giving away a bounty to the fastest / most practical. – Dan W May 20 '15 at 14:55
  • @DanW or just vote to close as duplicate as now you know what you trying to do - "c# sparse array" would give you couple answers like http://stackoverflow.com/questions/3798178/implementing-a-sparse-array-in-c-sharp-fastest-way-to-map-integer-to-a-specifi – Alexei Levenkov May 20 '15 at 14:57
  • 1
    @AlexeiLevenkov I will add that comment, but I don't think that the goal was a sparse array, it was a random integer indexer. Even one of the links in the answer you linked uses a `HashTable` as the backing store, which isn't a far cry from this type of solution. – Ron Beyer May 20 '15 at 15:03
  • 'Fraid my results weren't so positive (MyCoolType being 15-100x slower than MyList). However, I tested for speed once the arrays were already resized to a given max. See what you think, and let me know if I have made any errors: http://pastebin.com/cbEn1TgA – Dan W May 20 '15 at 19:14
  • Also I tested your code comparison but with less memory (20 million max size, instead of 2 billion), and 10M insertions took 1.5s in MyList and 4.5s in CoolType. And looking up 10M random elements took 1.75s in MyList and 3.3s in CoolType. Here is your version (but edited slightly): http://pastebin.com/Qbkwq8Ge – Dan W May 20 '15 at 19:41
  • Pre-allocating everything kind of defeats the purpose of your original question, at that point neither option is better over an array. As far as your results go I'm not sure why the difference, running your code on my system gives comparable results between the two, and when working with *really* large indicies, the dictionary wins out by 2-1 (your same test just more elements). – Ron Beyer May 20 '15 at 19:55
  • Fair point. I only [tested that](http://pastebin.com/cbEn1TgA) to see what performance was like when the array wasn't constantly being resized. Can you try removing the 3 lines: `s[maxArraySize - 1] = 999;ls[maxArraySize - 1] = 999; n[maxArraySize - 1] = 999;` from [my version](http://pastebin.com/cbEn1TgA). Now it's not pre-allocated, but on my PC, MyList still runs almost 100x faster than MyCoolType. – Dan W May 20 '15 at 20:34
0

You can define an extension method on List:

public static class ExtensionMethods {
    public static void Set<T>(this List<T> list, int index, T element) {
        if (index < list.Count) {
            list[index] = element;
        } else {
            for (int i = list.Count; i < index; i++) {
                list.Add(default(T));
            }
            list.Add(element);
        }
    }
}

and call list.Set(12, 1024) if you want the 12th element to be 1024.

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
  • *"I'd like to be able to keep the simple square bracket syntax (rather than using a verbose method)"* – Rawling May 20 '15 at 14:34
  • Ah, I might have missed that (or it was edited later). Then go for Joel Coehoorn's solution. – Glorfindel May 20 '15 at 14:35
  • I think you have your condition backwards. You'd want `index < list.Count` when you do `list[index] = element`. – juharr May 20 '15 at 14:36
0

Here is your pi

       static public List<int> AddToList(int index,int value,  List<int> input)
        {
            if (index >= input.Count)
            {
                int[] temparray = new int[index - input.Count + 1];
                input.AddRange(temparray);
            }
            return (input[index] = value);
        }
jdweng
  • 33,250
  • 2
  • 15
  • 20