How to get sub array without cloning

Question

I know in C# we can always get the sub-array of a given array by using Array.Copy() method. However, this will consume more memory and processing time which is unnecessary in read-only situation. For example, I'm writing a heavy load network program which exchanges messages with other nodes in the cluster very frequently. The first 20 bytes of every message is the message header while the rest bytes make up the message body. Therefore, I will divide the received raw message into header byte array and body byte array in order to process them separately. However, this will obviously consume double memory and extra time. In C, we can easily use a pointer and assign offset to it to access different parts of the array. For instance, in C language, if we have a char a[] = "ABCDEFGHIJKLMN", we can declare a char* ptr = a + 3 to represent the array DEFGHIJKLMN.

Is there a way to accomplish this in C#?

Might be interested in `unsafe`.. It's easy to get an Enumerable projection, but not another array "without a copy". — user2864740, Jul 17 '15 at 03:40
*"However, this will obviously consume double memory"* I don't see why, unless you keep two copies in memory. After you've divided it up, set the original one to null or let it go out of scope. — Ron Beyer, Jul 17 '15 at 03:40
Look into LINQ's Skip() and Take () extension methods. Don't worry about the array itself and learn to use IEnumerable. — Jonathon Reinhart, Jul 17 '15 at 03:42
You could define your own little type to wrap a target array + offset, and pass through accesses to the array with the offset added. — Blorgbeard, Jul 17 '15 at 03:46
`unsafe` or create your own array that doesn't copy instance just reference them. — Thus Spoke Nomad, Jul 17 '15 at 03:48
I'm also wondering why you need this? C#/.Net have GarbageCollector that does memory management for you. — Thus Spoke Nomad, Jul 17 '15 at 03:53
@MonoLightTech Yes, the CLR will do garbage collection at runtime, but in some cases this is not ideal. The CLR uses non-deterministic object destruction, so you can't know exactly when an object's memory will actually be reclaimed. I've seen large objects stay in memory until the system is idle or other opportune time, even if an object has long since lost all of its references. — PC Luddite, Jul 17 '15 at 04:00
This may help you: [Memory-Mapped Files](https://msdn.microsoft.com/en-us/library/dd997372%28v=vs.110%29.aspx) — M.kazem Akhgary, Jul 17 '15 at 04:18

gastonmancini · Accepted Answer · 2015-07-17T04:42:26.790

8

You might be interested in ArraySegments or unsafe.

ArraySegments delimits a section of a one-dimensional array.

Check ArraySegments in action

ArraySegments usage example:

 int[] array = { 10, 20, 30 };

 ArraySegment<int> segment = new ArraySegment<int>(array, 1, 2);
 // The segment contains offset = 1, count = 2 and range = { 20, 30 }

Unsafe define an unsafe context in which pointers can be used.

Unsafe usage example:

    int[] a = { 4, 5, 6, 7, 8 };

    unsafe
    {
        fixed (int* c = a)
        {
            // use the pointer
        }
    }

edited Jul 17 '15 at 04:42

answered Jul 17 '15 at 04:09

gastonmancini

1,102
8
19

Note that in .NET 4.5 (Visual Studio 2012) was improved, see [what is the use of `ArraySegment` class?](http://stackoverflow.com/questions/4600024/) (but it is not a `class`, it is a `struct` that now implements interfaces). – Jeppe Stig Nielsen Sep 09 '16 at 08:32

score 1 · Answer 2 · answered Jul 17 '15 at 03:48

1

First of all you must consider this as a premature optimization.

But you may use several ways to reduce memory consumption, if you sure you really need it:

1) You may use Flyweight pattern https://en.wikipedia.org/wiki/Flyweight_pattern to pool duplicated resources.

2) You may try to use unsafe directive and manual pointer management.

3) You may just switch to C for this functionality and just call native code from your C# program.

From my experience memory consumption for short-lived objects is not a big problem and I'd just write code with flyweight pattern and profile application afterwards.

answered Jul 17 '15 at 03:48

Mikhail Aksenov

561
6
11

"call native code from your C# program" shouldn't the best suggestion. For most cases, external native calls (DllImport) causes some problems and you can't debug it runtime... – Thus Spoke Nomad Jul 17 '15 at 03:50
I understand this, and my first suggestion was 'don't think about this memory consumption at all'. Moreover I prefer use logging instead for debugger, so it's not a big deal to see what's going on and scale this approach to full-featured logging system for your application. – Mikhail Aksenov Jul 17 '15 at 03:54

score 0 · Answer 3 · answered Jul 17 '15 at 04:16

Assuming you have a Message wrapper class in C#? Why not just add a property on it called header that returns the top 20 bytes.

You can easily accomplish this using skip and take suggested by Jonathon Reinhart above if you have the entire initial array in a memory array, but it sounds like you may have it in a network stream, which means the property might be a little more involved by doing a read of the initial 20 bytes from the the stream.

Something along the lines of:

class Message
{
    private readonly Stream _stream;
    private byte[] _inMemoryBytes;

    public Message(Stream stream)
    {
        _stream = stream;
    }

    public IEnumerable<byte> Header
    {
        get
        {
            if (_inMemoryBytes.Length >= 20)
                return _inMemoryBytes.Take(20);

            _stream.Read(_inMemoryBytes, 0, 20);
            return _inMemoryBytes.Take(20);
        }
    }

    public IEnumerable<byte> FullMessage
    {
        get
        {
            // Read and return the whole message. You might want amend to data already read.
        }
    }
}

How to get sub array without cloning

3 Answers3

Linked

Related