Prepend to a C# Array

Question

Given a populated byte[] values in C#, I want to prepend the value (byte)0x00 to the array. I assume this will require making a new array and adding the contents of the old array. Speed is an important aspect of my application. What is the best way to do this?

-- EDIT --

The byte[] is used to store DSA (Digital Signature Algorithm) parameters. The operation will only need to be performed once per array, but speed is important because I am potentially performing this operation on many different byte[]s.

Answer 1

If you are only going to perform this operation once then there isn't a whole lot of choices. The code provided by Monroe's answer should do just fine.

byte[] newValues = new byte[values.Length + 1];
newValues[0] = 0x00;                                // set the prepended value
Array.Copy(values, 0, newValues, 1, values.Length); // copy the old values

If, however, you're going to be performing this operation multiple times you have some more choices. There is a fundamental problem that prepending data to an array isn't an efficient operation, so you could choose to use an alternate data structure.

A LinkedList can efficiently prepend data, but it's less efficient in general for most tasks as it involves a lot more memory allocation/deallocation and also looses memory locallity, so it may not be a net win.

A double ended queue (known as a deque) would be a fantastic data structure for you. You can efficiently add to the start or the end, and efficiently access data anywhere in the structure (but you can't efficiently insert somewhere other than the start or end). The major problem here is that .NET doesn't provide an implementation of a deque. You'd need to find a 3rd party library with an implementation.

You can also save yourself a lot when copying by keeping track of "data that I need to prepend" (using a List/Queue/etc.) and then waiting to actually prepend the data as long as possible, so that you minimize the creation of new arrays as much as possible, as well as limiting the number of copies of existing elements.

You could also consider whether you could adjust the structure so that you're adding to the end, rather than the start (even if you know that you'll need to reverse it later). If you are appending a lot in a short space of time it may be worth storing the data in a List (which can efficiently add to the end) and adding to the end. Depending on your needs, it may even be worth making a class that is a wrapper for a List and that hides the fact that it is reversed. You could make an indexer that maps i to Count-i, etc. so that it appears, from the outside, as though your data is stored normally, even though the internal List actually holds the data backwards.

Answer 2

Ok guys, let's take a look at the perfomance issue regarding this question. This is not an answer, just a microbenchmark to see which option is more efficient.

So, let's set the scenario:

• A byte array of 1,000,000 items, randomly populated
• We need to prepend item 0x00

We have 3 options:

1. Manually creating and populating the new array
2. Manually creating the new array and using Array.Copy (@Monroe)
3. Creating a list, loading the array, inserting the item and converting the list to an array

Here's the code:

    byte[] byteArray = new byte[1000000];

    for (int i = 0; i < byteArray.Length; i++)
    {
        byteArray[i] = Convert.ToByte(DateTime.Now.Second);
    }

    Stopwatch stopWatch = new Stopwatch();

    //#1 Manually creating and populating a new array;

    stopWatch.Start();

    byte[] extendedByteArray1 = new byte[byteArray.Length + 1];

    extendedByteArray1[0] = 0x00;

    for (int i = 0; i < byteArray.Length; i++)
    {
        extendedByteArray1[i + 1] = byteArray[i];
    }

    stopWatch.Stop();
    Console.WriteLine(string.Format("#1: {0} ms", stopWatch.ElapsedMilliseconds));
    stopWatch.Reset();

    //#2 Using a new array and Array.Copy

    stopWatch.Start();

    byte[] extendedByteArray2 = new byte[byteArray.Length + 1];
    extendedByteArray2[0] = 0x00;                                
    Array.Copy(byteArray, 0, extendedByteArray2, 1, byteArray.Length);

    stopWatch.Stop();
    Console.WriteLine(string.Format("#2: {0} ms", stopWatch.ElapsedMilliseconds));
    stopWatch.Reset();

    //#3 Using a List

    stopWatch.Start();

    List<byte> byteList = new List<byte>();
    byteList.AddRange(byteArray);
    byteList.Insert(0, 0x00);

    byte[] extendedByteArray3 = byteList.ToArray();

    stopWatch.Stop();
    Console.WriteLine(string.Format("#3: {0} ms", stopWatch.ElapsedMilliseconds));
    stopWatch.Reset();

    Console.ReadLine();

And the results are:

#1: 9 ms
#2: 1 ms
#3: 6 ms

I've run it multiple times and I got different numbers, but the proportion is always the same: #2 is always the most efficient choice.

My conclusion: arrays are more efficient then Lists (although they provide less functionality), and somehow Array.Copy is really optmized (would like to understand that, though).

Any feedback will be appreciated.

Best regards.

PS: this is not a swordfight post, we are at a Q&A site to learn and teach. And learn.

Answer 3

The easiest and cleanest way for .NET 4.7.1 and above is to use the side-effect free Prepend().

Adds a value to the beginning of the sequence.

Example

// Creating an array of numbers
var numbers = new[] { 1, 2, 3 };

// Trying to prepend any value of the same type
var results = numbers.Prepend(0);

// output is 0, 1, 2, 3
Console.WriteLine(string.Join(", ", results ));

Answer 4

As you surmised, the fastest way to do this is to create new array of length + 1 and copy all the old values over.

If you are going to be doing this many times, then I suggest using a List<byte> instead of byte[], as the cost of reallocating and copying while growing the underlying storage is amortized more effectively; in the usual case, the underlying vector in the List is grown by a factor of two each time an addition or insertion is made to the List that would exceed its current capacity.

...

byte[] newValues = new byte[values.Length + 1];
newValues[0] = 0x00;                                // set the prepended value
Array.Copy(values, 0, newValues, 1, values.Length); // copy the old values

Answer 5

When I need to append data frequently but also want O(1) random access to individual elements, I'll use an array that is over allocated by some amount of padding for quickly adding new values. This means you need to store the actual content length in another variable, as the array.length will indicate the length + the padding. A new value gets appended by using one slot of the padding, no allocation & copy are necessary until you run out of padding. In effect, allocation is amortized over several append operations. There are speed space trade offs, as if you have many of these data structures you could have a fair amount of padding in use at any one time in the program.

This same technique can be used in prepending. Just as with appending, you can introduce an interface or abstraction between the users and the implementation: you can have several slots of padding so that new memory allocation is only necessary occasionally. As some above suggested, you can also implement a prepending interface with an appending data structure that reverses the indexes.

I'd package the data structure as an implementation of some generic collection interface, so that the interface appears fairly normal to the user (such as an array list or something).

(Also, if removal is supported, it's probably useful to clear elements as soon as they are removed to help reduce gc load.)

The main point is to consider the implementation and the interface separately, as decoupling them gives you the flexibility to choose varied implementations or to hide implementation details using a minimal interface.

There are many other data structures you could use depending on the applicability to your domain. Ropes or Gap Buffer; see What is best data structure suitable to implement editor like notepad?; Trie's do some useful things, too.

Answer 6

I know this is a VERY old post but I actually like using lambda. Sure my code may NOT be the most efficient way but its readable and in one line. I use a combination of .Concat and ArraySegment.

 string[] originalStringArray = new string[] { "1", "2", "3", "5", "6" };
 int firstElementZero = 0;
 int insertAtPositionZeroBased = 3;
 string stringToPrepend = "0";
 string stringToInsert = "FOUR"; // Deliberate !!!

 originalStringArray = new string[] { stringToPrepend }
            .Concat(originalStringArray).ToArray();

 insertAtPositionZeroBased += 1; // BECAUSE we prepended !!
 originalStringArray = new ArraySegment<string>(originalStringArray, firstElementZero, insertAtPositionZeroBased)       
                .Concat(new string[] { stringToInsert })
                .Concat(new ArraySegment<string>(originalStringArray, insertAtPositionZeroBased, originalStringArray.Length - insertAtPositionZeroBased)).ToArray();

Answer 7

The best choice depends on what you're going to be doing with this collection later on down the line. If that's the only length-changing edit that will ever be made, then your best bet is to create a new array with one additional slot and use Array.Copy() to do the rest. No need to initialize the first value, since new C# arrays are always zeroed out:

byte[] PrependWithZero(byte[] input)
{
    var newArray = new byte[input.Length + 1];
    Array.Copy(input, 0, newArray, 1, input.Length);
    return newArray;
}

If there are going to be other length-changing edits that might happen, the most performant option might be to use a List<byte> all along, as long as the additions aren't always to the beginning. (If that's the case, even a linked list might not be an option that you can dismiss out of hand.):

var list = new List<byte>(input);
list.Insert(0, 0);

Answer 8

I am aware this is over 4-year-old accepted post, but for those who this might be relevant Buffer.BlockCopy would be faster.