Read all available bytes from TCP Socket (unknown byte count)

Question

I am having Problems useing the Indy TIdTCPClient. I want to call a function, everytime if there is Data available on the socket. For this I have a Thread calling IdTCPClient->Socket->Readable(100). The function itself looks like this:

TMemoryStream *mStream = new TMemoryStream;
int len = 0;
try
{
    if(!Form1->IdTCPClient2->Connected())
        Form1->IdTCPClient2->Connect();
    mStream->Position = 0;
    do
    {
        Form1->IdTCPClient2->Socket->ReadStream(mStream, 1);
    }
    while(Form1->IdTCPClient2->Socket->Readable(100));

    len = mStream->Position;
    mStream->Position = 0;
    mStream->Read(Buffer, len);
}catch(Exception &Ex) {
    Form1->DisplaySSH->Lines->Add(Ex.Message);
    Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;

It will not be called directly within the thread, but the thread triggers an event, which is calling this function. Which means I am using Readable(100) twice, without reading data in betwee. So since I dont know how many bytes I have to read I thought I can read one byte, check if there is more available and then read another byte. The Problem here is that the do while loop doesnt loop, it just runs once. I am guessing that Readable does not quite wokt the way I need it to. Is there any other way to receive all the bytes available in the Socket?

What do you mean by "all available bytes"? Do you mean all the bytes that happen to be available at this particular instant? What's your actual problem? Why do you think you need to read all available bytes? — David Schwartz, Nov 01 '18 at 07:38
Yes I do. I am receiving encrypted Data, thats why I can not read the length of one package. The problem is that I need to read data from the socket without knowing the length of the data I need to read. — Adi, Nov 01 '18 at 07:42
So what are you going to do with the bytes when you get them? They still could be a partial message. They could be part of one message and part of another. You'll need some pretty complex code to extract the messages from the data you receive. — David Schwartz, Nov 01 '18 at 07:44
I need to put it in the buffer of the event. The event is part of an SSH Client. Well honestly I havent thought about that much.... — Adi, Nov 01 '18 at 07:48
Then maybe [this question](https://stackoverflow.com/questions/10881685/indy-tcp-read-data-in-a-loop) will help. — David Schwartz, Nov 01 '18 at 07:52
Actually yes it does help a lot, The only problem is that the `ReadFromStack()` function seems to be removed, or maybe renamed? — Adi, Nov 01 '18 at 08:11
`TIdTCPConnection::ReadFromStack()` applies to Indy 9. In Indy 10, it was replaced with `TIdIOHandler::ReadFromSource()` and `TIdIOHandler::CheckForDataOnSouce()`. — Remy Lebeau, Nov 01 '18 at 17:50

score 0 · Accepted Answer · answered Nov 01 '18 at 18:10

You should not be using Readable() directly in this situation. That call reports whether the underlying socket has pending unread data in its internal kernel buffer. That does not take into account that the TIdIOHandler may already have unread data in its InputBuffer that is left over from a previous read operation.

Use the TIdIOHandler::CheckForDataOnSource() method instead of TIdIOHandler::Readable():

TMemoryStream *mStream = new TMemoryStream;
try
{
    if (!Form1->IdTCPClient2->Connected())
        Form1->IdTCPClient2->Connect();

    mStream->Position = 0;

    do
    {
        if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
        {
            if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
                break;
        }

        Form1->IdTCPClient2->IOHandler->ReadStream(mStream, Form1->IdTCPClient2->IOHandler->InputBuffer->Size, false);

        /* alternatively:
        Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToStream(mStream);
        */
    }
    while (true);

    // use mStream as needed...
}
catch (const Exception &Ex) {
    Form1->DisplaySSH->Lines->Add(Ex.Message);
    Form1->DisplaySSH->GoToTextEnd();
}
delete mStream;

Or, you can alternatively use TIdIOHandler::ReadBytes() instead of TIdIOHandler::ReadStream(). If you set its AByteCount parameter to -1, it will return only the bytes that are currently available (if the InputBuffer is empty, ReadBytes() will wait up to the ReadTimeout interval for the socket to receive any new bytes) ¹:

try
{
    if (!Form1->IdTCPClient2->Connected())
        Form1->IdTCPClient2->Connect();

    TIdBytes data;

    do
    {
        if (Form1->IdTCPClient2->IOHander->InputBufferIsEmpty())
        {
            if (!Form1->IdTCPClient2->IOHander->CheckForDataOnSource(100))
                break;
        }

        Form1->IdTCPClient2->IOHandler->ReadBytes(data, -1, true);

        /* alternatively:
        Form1->IdTCPClient2->IOHandler->InputBuffer->ExtractToBytes(data, -1, true);
        */
    }
    while (true);

    // use data as needed...
}
catch (const Exception &Ex) {
    Form1->DisplaySSH->Lines->Add(Ex.Message);
    Form1->DisplaySSH->GoToTextEnd();
}

^{1: make sure you are using an up-to-date snapshot of Indy 10. Prior to Oct 6 2016, there was a logic bug in ReadBytes() when AByteCount=-1 that didn't take the InputBuffer into account before checking the socket for new bytes.}

Thanks, that works like a charm. One more thing, the `CheckForDataOnSource()` adds data to the input buffer and does not replace it right? — Adi, Nov 02 '18 at 08:10
@Adi new data is always appended to the end of the `InputBuffer`, preserving existing data. — Remy Lebeau, Nov 02 '18 at 15:38

Read all available bytes from TCP Socket (unknown byte count)

1 Answers1