1

I want to copy a section of a memory stream into an array of doubles. Something like:

MemoryStream stream = new(File.ReadAllBytes(pathToFileWithBinaryData));
int arrayLength = stream.Length/sizeof(double);
double[] array = new double[arrayLength];
byte[] buffer = new byte[sizeof(double)];
int i=0;
while(stream.Position < stream.Length)
{
  stream.Read(buffer, 0, sizeof(double));
  array[i] = BitConvert.ToDouble(buffer);
}

But, I don't want to copy the values from buffer to array one at a time. The C/C++ programmer in me is inclined to tackle it this way:

byte[] buffer = /* the memory address of the variable 'array' ??? */
stream.Read(buffer, 0, stream.Length);

and then I would have all the values in array because it is the memory that stream.Read copied into. Is there a way to do this? Then I could use stream.ReadAsync and be very happy.

dmedine
  • 1,430
  • 8
  • 25
  • 2
    Does this answer your question? [byte array to double conversion in c#](https://stackoverflow.com/questions/3421709/byte-array-to-double-conversion-in-c-sharp) and [How to convert a byte array to double array in C#?](https://stackoverflow.com/questions/7832120/how-to-convert-a-byte-array-to-double-array-in-c) –  Aug 06 '21 at 05:35
  • To be clear, you're expecting the source data to contain a sequence of bytes that correctly represents a `double`, exactly as is? You're explicitly *not* trying to create a `double` that has the same numeric value as a given `byte`? – Karl Knechtel Aug 06 '21 at 05:38
  • No need of [a stream and a loop](https://learn.microsoft.com/dotnet/api/system.bitconverter.todouble?view=net-5.0#System_BitConverter_ToDouble_System_Byte___System_Int32_), or use "memory entanglement" with [FieldOffsetAttribute](https://learn.microsoft.com/dotnet/api/system.runtime.interopservices.fieldoffsetattribute). –  Aug 06 '21 at 05:38
  • @KarlKnechtel yes, I know the data is doubles. I expect to use 8 bytes to represent one double. – dmedine Aug 06 '21 at 05:47
  • So if your byte is size 80 you want an array of 10 doubles? Any endianness/order considerations we have to know about? – Caius Jard Aug 06 '21 at 05:49
  • 1
    @OlivierRogier the first one seems quite promising! I think `Buffer.BlockCopy` is what I want. This seems to be like `memcpy`. – dmedine Aug 06 '21 at 05:50
  • @CaiusJard No endianness considerations. I know the doubles are encoded the way I want. – dmedine Aug 06 '21 at 05:51
  • @OlivierRogier but, it would be better if I didn't need an intermediate buffer (`Bufffer.BlockCopy` method--which copies twice). I would rather read directly into the memory allocated to `array`. – dmedine Aug 06 '21 at 05:55
  • @dmedine [Array.Copy vs Buffer.BlockCopy](https://stackoverflow.com/questions/1389821/array-copy-vs-buffer-blockcopy) • [memcpy function in c#](https://stackoverflow.com/questions/2996487/memcpy-function-in-c-sharp) | To avoid copying and doing direct access based on the type you are looking for, try the mentioned "memory entanglement" using FieldOffsetAttribute: it is like [C unions](https://www.tutorialspoint.com/cprogramming/c_unions.htm). –  Aug 06 '21 at 06:03
  • @OlivierRogier right, but I want to copy right out of the stream. Maybe your FiledOffsetAttribute is a path to this? Something like a C union is definitely a solution. But it is not at all clear to me from the MSDN documentation you've linked to how that would work. – dmedine Aug 06 '21 at 06:08
  • @dmedine Does [read the file in a byte array with a single method call](https://stackoverflow.com/questions/2030847/best-way-to-read-a-large-file-into-a-byte-array-in-c), having a [union-like to read doubles](https://stackoverflow.com/questions/839903/c-sharp-structlayout-fieldoffset-and-indexing-in-arrays), do the job? –  Aug 06 '21 at 06:36

1 Answers1

1

Thanks to the hints from @OlivierRogier, I found the solution that I was looking for. Here is an example program:

using System;
using System.IO;
using System.Runtime.InteropServices;

namespace MemoryEntangle
{
    [StructLayout(LayoutKind.Explicit)]
    public struct MemoryArea 
    {
        [FieldOffset(0)]
        public byte[] _buffer;

        [FieldOffset(0)]
        public double[] _array;
    }

    class Program
    {
        static void Main()
        {
            const int count = 10;
            MemoryArea ma = new();
            ma._buffer = new byte[count * sizeof(double)];
            //mab._array = new double[count];
            MemoryStream memoryStream = new();
            for (int i = 0; i < count; i++)
            {
                memoryStream.Write(BitConverter.GetBytes((double)i * .1));
                Console.WriteLine("{0}", (double)i * .1);
            }
            memoryStream.Position = 0;
            memoryStream.Read(ma._buffer, (int)memoryStream.Position, (int)memoryStream.Length);
            for (int i = 0; i < count; i++)
                Console.WriteLine("{0}", ma._array[i]);
        }
    }
}

Output:

0
0.1
0.2
0.30000000000000004
0.4
0.5
0.6000000000000001
0.7000000000000001
0.8
0.9
0
0.1
0.2
0.30000000000000004
0.4
0.5
0.6000000000000001
0.7000000000000001
0.8
0.9
dmedine
  • 1,430
  • 8
  • 25
  • I am not getting weird values like `0.300000000000000004` but `0.3` with .NET Framework 4.7.2 on Windows 10-x64 Ryzen 7-5800X: what is your machine, OS and target .NET implementation? Tips: `(double)` cast is useless since `i` is already a `double` as well as the right operand is. –  Aug 09 '21 at 02:56
  • Huh. `i` looks like an `int` to me ;-). I am using .NET 5.0 on Windows 10 Lenovo T560. The roundoff error struck me as strange too, which is why I printed out the values twice. I wanted to know it wasn't an artifact from the `union`. – dmedine Aug 09 '21 at 03:27
  • 1
    Another weird thing is that if I pass `_array` to a function that takes `double[]` as a parameter, it is parsed as if it is a `byte[]` and throws no warning or error. I guess that is what this: https://stackoverflow.com/questions/839903/c-sharp-structlayout-fieldoffset-and-indexing-in-arrays is about. I cannot use unsafe code, so I may have to copy one by one anyway. – dmedine Aug 09 '21 at 03:29
  • Indeed, i is an int, but multiplicated, I misread :) –  Aug 09 '21 at 03:50
  • 1
    In any case, the compiler does not complain when I remove the cast, so you are right that it is not necessary. – dmedine Aug 09 '21 at 03:52
  • I just tested this double array with the display loop in a method instead of Main and it works only if I use the count instead of the double array Length that is 80. `static void test(double[] array) { for ( int i = 0; i < 10 /* don't use array.Length that is 80! */; i++ ) Console.WriteLine("{0}", array[i]); }`. But indeed if I inspect array VS tells it is a byte[] and I don't understand even reading the link. Even ma._array is indicated as byte[] and not double[], being Length of 80! Thus this is not really like C unions as i remember but it works & need to keep track of the real length (10). –  Aug 09 '21 at 04:16
  • If I write `ma._array = new double[count];` the stream read crashes because the byte array is length of 10! That's not logical... –  Aug 09 '21 at 04:17
  • I would say that this is wrong in the C#/.Net specification. In no universe should an object that is `double[]` appear as `byte[]`. This is not how C unions work, but then again, C doesn't have boxing. Everything is just a pointer and the compiler knows about array types and automagically adjusts increment widths based on context. I have no idea, but it could be that this whole mechanism is a hack that will never work smoothly. It might be worth while asking Microsoft about this. – dmedine Aug 10 '21 at 00:33