Infinite Loop in Delphi Procedure

Question

I'm having a weird issue on using Delphi's TMemoryStream (or TFileStream for that matter). While reading a part of the stream into a byte array. Here's some code as an example.

procedure readfromstream();
var
     ms : TMemoryStream;
     buffer : array of byte;
     recordSize : Integer;
begin
  try
  begin
     ms := TMemeoryStream.Create();
     ms.LoadFromFile(<some_path_to_a_binary_file>);

     while ms.Position < ms.Size do
     begin
         buffer := nil;
         SetLength(buffer, 4);
         ms.ReadBuffer(buffer, 4);
         move(buffer[0], recordSize, 4);

         SetLength(buffer, recordSize);
         ms.Position := ms.Position - 4;           // Because I was having issues trying to read the rest of the record into a specific point in the buffer
         FillChar(buffer, recordSize, ' ');
         ms.ReadBuffer(buffer, recordSize);        // Issue line ???

         // Create the record from the buffer
     end;
  finally
  begin
     ms.Free();
  end;
end;

procedure is called as,

// Some stuff happens before it

readfromstream();

// Some stuff happens after it

on debugging, I can see that it reads the stream into the buffer and the record is stored in memory appropriately. The procedure then exits normally and the debugger steps out of the procedure, but I end up straight back into the procedure and it repeats.

By forcing the procedure to exit prematurely I believe the issue involves the ms.ReadBuffer(buffer, recordSize); but I don't see why it would cause the issue.

This procedure is called only once. My test data has only one entry/data. Any help would be greatly appreciated.

Answer 1

FillChar(buffer, recordSize, ' ');

Here you are overwriting the dynamic array variable, a pointer, rather than writing to the content of the array. That causes a memory corruption. Pretty much anything goes at that point.

The call to FillChar is needless anyway. You are going to read into the entire array anyway. Remove the call to FillChar.

For future reference, to do that call correctly, you write it like this:

FillChar(Pointer(buffer)^, ...);

or

FillChar(buffer[0], ...);

I prefer the former since the latter is subject to range errors when the array length is zero.

And then

ms.ReadBuffer(buffer, recordSize);

makes the exact same mistake, writing to the array variable rather than the array, and thus corrupting memory.

That should be

ms.ReadBuffer(Pointer(buffer)^, recordSize);

or

ms.ReadBuffer(buffer[0], recordSize);

---

The first 4 lines inside the loop are clumsy. Read directly into the variable:

ms.ReadBuffer(recordSize, SizeOf(recordSize));

I recommend that you perform some sanity checks on the value of recordSize that you read. For instance, any value less than 4 is clearly an error.

There's not a lot of point in moving the stream pointer back and reading again. You can copy recordSize into the first 4 bytes and the array and then read the rest.

Move(recordSize, buffer[0], SizeOf(recordSize));
ms.ReadBuffer(buffer[SizeOf(recordSize)], recordSize - SizeOf(recordSize));

A memory stream also seems wasteful. Why read the entire file into memory? That's going to place stress on your address space for large files. Use a buffered file stream.

Letting the caller allocate the stream would give more flexibility to the caller. They could then read from any type of stream and not be constrained to use a disk file.

Your try/finally block is wrong. You must acquire the resource immediately before the try. As you have it, an exception in the constructor leads to you calling Free on an uninitialized variable.

A better version might be:

procedure ReadFromStream(Stream: TStream);
var
  buffer: TArray<byte>;
  recordSize: Integer;
begin
  while Stream.Position < Stream.Size do
  begin
    Stream.ReadBuffer(recordSize, SizeOf(recordSize));     
    if recordSize < SizeOf(recordSize) then
      raise ...;

    SetLength(buffer, recordSize);
    Move(recordSize, buffer[0], SizeOf(recordSize));
    if recordSize > SizeOf(recordSize) then
      Stream.ReadBuffer(buffer[SizeOf(recordSize)],
        recordSize - SizeOf(recordSize));

    // process record
  end;
end;       

Answer 2

Sorry I can't add a comment, being a newb and all :) This reply is based on my understanding of Clayton's code in light of his comment with the recordSize values.

The reason David's code is looping is probably that you are interpreting each four byte "block" is a number. I'll assume your first Stream.Readbuffer is correct and that the first four bytes in the file is a length.

Now, unless I'm mistaken, I expect the recordSize will usually be greater than SizeOf(recordSize), which I think should be 4 (the size of an int). Nevertheless, this line is meaningless here.

The SetLength is correct, given my previous assumption.

Now your Move is where the story hits a snag: you haven't read anything since you read the length! So before the move, you should have: bytesRead := Stream.Readbuffer(Pointer(buffer)^, recordSize);

Now you can check for EOF: if bytesRead <> recordSize then raise...;

...and move the buffer somewhere (if you wish): Move(buffer[0], dest[0], recordSize);

And you are positioned to read the next recordSize value.

Repeat until EOF.