C# - Stream.Read offset is working incorrectly

Question

I have a test code like below. I am reading from stream, offsetting by 2 positions, and then taking next 2 bytes. I would hope that result would be an array with 2 elements. This does not work though - offset is completely ignored, and full sized array is always returned, with only offset blocks having values. But this means my result table is still very large, it just has a lot of unwanted zeroes

How can I rework below code, so that file.Read() returns only an array of 2 bytes instead of 10 when length = 2 and offset = 2? In real world scenario I am dealing with large files (>2gigs) so filtering out the result array is not an option.

Edit: As the issue is unclear - below code requires me to always define output array that is the same size as the stream. Instead I would like to have an output that is of size of length (in below example I would like to have var buffer = new byte[2], but that will throw an exception because file.Read ignores offset and length and always returns 10 elements (with only 2 of them being read, rest is dummy zeroes).

        private byte[] GetFilePart(int length, int offset)
        {
            //build some dummy content
            var content = new byte[10];
            for (int i = 0; i<10; i++)
            {
                content[i] = 1;
            }

            //read the data from content
            var buffer = new byte[10];
            using (Stream file = new MemoryStream(content))
            {
                file.Read(buffer, offset, length);
            }
            return buffer;
        }

I don't follow your question -- the screenshot looks correct. It read 2 bytes from the stream, and wrote them at offset 2 in `buffer` (so indexes 2 and 3 in `buffer` were written to). What did you expect to happen? — canton7, Jun 17 '21 at 12:40
The `Stream.Read` don't create a new array, it populates the one you passed. So, if you want the result array to have size of 2, you need to pass a array with size of 2. Also, [check the documentation](https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.read?view=net-5.0#System_IO_Stream_Read_System_Byte___System_Int32_System_Int32_) — Magnetron, Jun 17 '21 at 12:46
@canton7 I would like to define var buffer = new byte[2]. But then the code fails as it's trying to always fill in 10 elements. — ojek, Jun 17 '21 at 13:00
@ojek I don't think it's that -- if `length` is 2, it will always read 2 bytes. If you have a 2-byte buffer, and try to read 2 bytes into it, offset by 2 (i.e. writing at indexes 2 and 3 in the buffer), then that will fail of course: you need a 4-byte buffer to be able to write to indexes 2 and 3 of that buffer: that's common sense. I think the confusion might be that `offset` is the offset into `buffer` that you want to start writing at, not the offset in `file` that you want to start reading at. As the doc says, "*The zero-based byte offset in buffer at which to begin storing the data*" — canton7, Jun 17 '21 at 13:01
Re your edit -- `stream.Read` doesn't *return* anything! You *pass in* a buffer. The buffer has a length of 10 because you declared it as length 10! If you want a buffer of length 2, then simply declare a buffer of length 2. As my previous comment says, obviously attempting to write to indexes 2 and 3 of a 2-element array will fail. Try this: `byte[] buffer = new byte[2]; buffer[2] = 1; buffer[3] = 2`, Surely it's clear that that will fail, for obvious reasons. That's the same reason why `byte[] buffer = new byte[2]; stream.Read(buffer, 2, 2)` will fail — canton7, Jun 17 '21 at 13:04

Caius Jard · Accepted Answer · 2021-06-17T13:03:40.557

Looks like it's working properly to me; maybe your confusion would clear a bit if you inited your content array with something like:

        for (int i = 1; i<=10; i++)
        {
            content[i-1] = i;
        }

then each byte would have a different number and the image would look like:

offset relates to where into buffer the Stream will write the bytes to (it reads from the start of content). It does not relate to what bytes are read out of content.

Imagine Read as being called WriteBytesInto(byte[] whatBuffer, int whereToStartWriting, int howManyBytesToWrite) - you provide the buffer it will write into and tell it where to start and how many to do

If you did this, having inited content to be incrementing numbers:

file.Read(buffer, 2, 3); //read 3 bytes from stream and write to buffer @ index 2
file.Read(buffer, 0, 2); //read 2 bytes from stream and write to buffer @ index 0

Your buffer would end up looking like:

4,5,1,2,3,0,0,0,0,0

The 1,2,3 having been written first, then the 4,5 written next

If you want to skip two bytes from the stream (i.e. read the 3rd and 4th byte from content, Seek() the stream or set its Position (or as canton7 advises in the commments, if the stream is not seekable, read and discard some bytes)

How can I rework below code, so that file.Read() returns only an array of 2 bytes instead of 10 when length = 2 and offset = 2?

Well, file.Read doesn't return an array at all; it modifies an array you give it. If you want a 2 byte array, give it a 2 byte array:

byte buf = new byte[2];
file.Read(buf, 0, buf.Length);

If you want to open a file, skip the first 7 bytes and then read bytes 8th and 9th into your length-of-2 byte array then:

byte buf = new byte[2];
file.Position = 7; //absolute skip to 8th byte
file.Read(buf, 0, buf.Length);

For more on seeking in streams see Stream.Seek(0, SeekOrigin.Begin) or Position = 0

(Note that not all streams can be seeked, check `Stream.CanSeek`. If the stream isn't seekable, you'll just have to read and discard any bytes you want to skip) — canton7, Jun 17 '21 at 12:47
It can, but they also said it was "dummy content", so I don't want to make any assumptions about their actual use-case! — canton7, Jun 17 '21 at 12:49
Also, OP confusion looks related to `buffer` size. You could edit your answer to address the fact that `Stream.Read` don't create a new array, but populate the one you pass to it. So, if OP want a buffer with size of 2, they must do `var buffer = new byte[2];` — Magnetron, Jun 17 '21 at 12:50
Thank you! your comment "offset relates to where into buffer the Stream will write the bytes to (it reads from the start of content). It does not relate to what bytes are read out of content" explains my confusion pretty well. I was thinking the offset is for the stream source, and you are correct, it's for the output buffer which all makes sense now, appreciate the clarification! — ojek, Jun 17 '21 at 13:08
@ojek when in doubt, always [check the documentation](https://learn.microsoft.com/en-us/dotnet/api/system.io.stream.read?view=net-5.0#System_IO_Stream_Read_System_Byte___System_Int32_System_Int32_). MS has a pretty decent documentation of their APIs. — Magnetron, Jun 17 '21 at 13:58

C# - Stream.Read offset is working incorrectly

1 Answers1