0

I am using Delphi XE6.

I have read all of the following:

Slow responses to TIdHTTP POSTs

Delphi: Why does IdHTTP.ConnectTimeout make requests slower?

Delphi TIdHTTP POST is very slow vs GET

And I know that relevent timing and performance factors include : 1. HTTPOptions 2. TIdHttp.Request options - content type, encoding, and especially connection timeout settings. 3. iCsslIOHandler.SSLOptions.Method should select the correct SSL version for authentication

But my TIdHttp.Post from Delphi Xe6 is still much slower than it should be, by a factor of 10x.

You can assume the posts are working - they are. No exception gets thrown, although there is no content in the log file I write out either.

Is it likely that using REST calls would make this faster, since http is not supposed to be RPC?

Does anyone have any tips or insights given the code below (iC prefixed variables are class scope, except for Mutexes which are not owned by the thread):

procedure TVinterClientCall<T , I>.InitialiseHttpClient;
var
  vKey           : String;
  vPair          : TPair<String , String>;
  vDebugString   : string;
  vSocketHandler : TIdSSLIOHandlerSocketOpenSSL;
  vParamValue    : String;
begin
  // Setup the Http Client
  iChttpClient := TIdHttp.Create;
  // System.RegisterExpectedMemoryLeak(iChttpClient);

  with iChttpClient do
  begin
    HandleRedirects := True;
    // Here we set the API and protocol content type
    Request.ContentType := iCContentType;
    Request.Accept      := iCContentType;


    ConnectTimeout     := 0;
    HTTPOptions        := [hoForceEncodeParams , hoKeepOrigProtocol];
    Request.Connection := 'keep-alive';

    // Logically, this should be redundant. There is no reason why this property should revert due to response
    // data or conditions. However, this is to be certain.
    if hoWaitForUnexpectedData in HTTPOptions then
    begin
      HTTPOptions := HTTPOptions - [hoWaitForUnexpectedData];
    end;

    //This just gets some header values from the interface, no problem...
    for vKey in iCCustomHeaderParams.Keys do
    begin
      // vParamValue is an out parameter
      iCCustomHeaderParams.TryGetValue(vKey , vParamValue);
      Request.CustomHeaders.AddValue(vKey , vParamValue);

    end;
  end;

  iCsslIOHandler                   := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
  iCsslIOHandler.SSLOptions.Method := sslvSSLv3; //**Is this right?**

  iChttpClient.IOHandler := iCsslIOHandler;

end;

// This is the currently used method.
procedure TVinterClientCall<T , I>.ExecuteCallMemStream;
const
  CRLF = #13#10;
var
  vStream                  : TMemoryStream;
  vParams                  : TMemoryStream;
  vResponse                : TMemoryStream;
  vRequestString           : string;
  vHeaderString            : string;
  vRequestStringStream     : TStringStream;
  vDisplayStreamDataString : String;
  vResponseData            : TStringStream;               // TidStream is Version 10.1.x. TMemoryStream may be better...
  vTimeStamps              : array [0 .. 1] of TDateTime; // Array of real;
  v64TimeStamps            : array [0 .. 1] of DWORD;     // int64;
  vDuration                : TDateTime;
  v64Duration              : DWORD;
  vHour                    : word;
  vMin                     : word;
  vSec                     : word;
  vMsec                    : word;
  vTimingLog               : TextFile;
  vBeforePostNow           : int64; // cardinal;
  vAfterPostNow            : int64; // cardinal;

  vPostTimes : TStringList;


begin

  vStream              := TMemoryStream.Create;
  vParams              := TMemoryStream.Create;
  iCResponse           := TMemoryStream.Create;
  vRequestStringStream := TStringStream.Create;
  vResponseData        := TStringStream.Create;
  vPostTimes           := TStringList.Create;

  InitialiseHttpClient();

  iCURL := iCEndPoint;


    try

      vRequestString := '{' + ProtocolParamsString + ',' + Method + ',' + DataParamString + '}';
      StringReplace(vRequestString , #13#10 , '' , [rfReplaceAll]);


      vRequestStringStream.WriteString(vRequestString);
      vDisplayStreamDataString := vRequestStringStream.DataString;

      iCRequestData := vDisplayStreamDataString;
      vParams.Write(vRequestString[1] , Length(vRequestString));

      with iChttpClient do
      begin
        try

          iCMxUtility.Acquire;
          v64TimeStamps[0] := timeGetTime(); // Now64();

          // vResponseData  := TStringStream.Create(Post(iCURL , vRequestStringStream));
          Post(iCURL , vRequestStringStream , vResponseData);

          v64TimeStamps[1] := timeGetTime(); // Now64();
          // vTimeStamps[1] := Now;
          iCMxUtility.Release;
          v64Duration := v64TimeStamps[1] - v64TimeStamps[0];
          // vDuration      := vTimeStamps[1] - vTimeStamps[0];

          // DecodeTime(vDuration , vHour , vMin , vSec , vMsec);

        except
          on E : Exception do
          begin
            iCMxUtility.Acquire;
            showmessage('Error encountered during POST: ' + E.Message);
            iCMxUtility.Release;
          end;

        end;
      end;

    finally


      iCMxLogFiles.Acquire;
      // Try to open the Test.txt file for writing to
      vPostTimes.SaveToFile('.\PostTimes_'+ StringReplace(TimeToStr(Now), ':', '', [rfReplaceAll]));

      iCMxLogFiles.Release;

      // Local variable scope Cleanup.
      FreeAndNil(vStream);
      FreeAndNil(vParams);
      FreeAndNil(iCResponse);
      FreeAndNil(vRequestStringStream);
      FreeAndNil(vResponseData);
      FreeAndNil(vPostTimes);

      ReleaseHttpClient;

    end;


end;
Community
  • 1
  • 1
Bruce Long
  • 713
  • 3
  • 9
  • 28
  • 1
    `**Is this right?***`: SSL Method `sslvSSLv3` looks a little dangerous – mjn Oct 28 '14 at 11:11
  • 1
    Are you sure it is Indy 9, not 10? – mjn Oct 28 '14 at 11:13
  • 2
    small tip: Do not use Showmessage in threads (not threadsafe), use MessageDlg() instead. Why are you using a critical section when you Post? Please provide an [MVCE](http://stackoverflow.com/help/mcve) so that we can verify... – whosrdaddy Oct 28 '14 at 11:56
  • 1
    What are the actual figures for GET and POST time in milliseconds? – mjn Oct 28 '14 at 12:10
  • 1
    @mjn For Delphi XE6, I would assume it's Indy 10. But OP may have copied some code from somewhere that was written for 9. – Jerry Dodge Oct 28 '14 at 16:17
  • 2
    @whosrdaddy: `MessageDlg()` is not thread-safe, either. Like `ShowMessage()`, `MessageDlg()` displays a `TForm`. If you really need a thread-safe popup message, use `Windows.MessageBox()` instead. – Remy Lebeau Oct 28 '14 at 18:10
  • 1
    You should profile the code to find out where the slowdown is actually occuring. If I had to guess, it is the `TStringStream` objects that are slowing things down, as `TStringStream` is a little slower in D2009+ then in earlier versions due to added `TBytes` and `String` handling logic. I would suggest using `TMemoryStream` instead. – Remy Lebeau Oct 28 '14 at 18:16
  • The Mutex on the call is a hangover from a log file write out that was there before after the call. @mjn: Yes Indy 10 my mistake on question. – Bruce Long Oct 28 '14 at 19:40
  • @Whosyr. and Remy It doesn't get invoked but thanks. Thanks for the tip Re TMemoryStream I will try that. By profile I take it you mean walkthough with the debugger. Two things: the timestamp values in the array are captured immediately each side of the code, and so nothing else happens between them. There are memory leaks in the code, but only one call is being executed from one thread out of the pool 2. When I try with a longer call the time stamps report 15000 + ms but it is clearly not that long on manual count - but much faster. That has me confused. – Bruce Long Oct 28 '14 at 19:45
  • @mjn **Is this right?***: SSL Method sslvSSLv3 looks a little dangerous because?? I am being lazy now - but I am not sure if you know something about it or are making a quip about the comment ;) – Bruce Long Oct 28 '14 at 19:48
  • Should I use REST? The API offers a REST JSON endpoint as well. I should not have thought any speed advantage was available unless it is the server that is slow. Slow server response seems. @Remy - which profiler do you use? – Bruce Long Oct 28 '14 at 19:51
  • 1
    SSLv3 is currently in the media because of the 'Poodle' security vulnerability. It is a 15+ years old standard iirc, TLS is the current standard for HTTPS. – mjn Oct 28 '14 at 20:25
  • 1
    @user3098527: REST is just HTTP, and Embarcadero's REST framework uses TIdHTTP internally. – Remy Lebeau Oct 28 '14 at 21:34
  • 1
    @user3098527: by "profile", I meant running the project in a profiler, like AQTime or similar, and let it actually clock the code in real-time and tell you what code it is actually spending its time on. Me personally, I don't use any profilers at all, actually. – Remy Lebeau Oct 28 '14 at 21:39
  • I am not sure a profiler will help much? The request is posted from a thread. There is only one thread doing one post and the timegettimes are directly before and after. – Bruce Long Oct 28 '14 at 21:42
  • `Post()` does a lot of work internally. A profiler would show you where the code is spending its time. Is it waiting on a read? Is it parsing strings slowly? Etc. That is what a profiler is good for. Putting calls to `timeGetTime()` around `Post()` does not offer useful info. – Remy Lebeau Nov 01 '14 at 15:17
  • Also, use a packet sniffer, such as Wireshark, to make sure the slowdown is not being caused by slow network traffic. If the server is slow to reply, that will make `Post()` take longer to exit. – Remy Lebeau Nov 01 '14 at 15:20
  • It would also allow you to compare the requests and responses between `TIdHTTP` and other REST components/libraries. Maybe the `TIdHTTP` requests are not as optimized as others, or maybe the server is performing extra redirects with `TIdHTTP` and not others. Try tweaking the `TIdHTTP.Request.UserAgent` to mimic popular browsers/libraries. – Remy Lebeau Nov 01 '14 at 15:28
  • Thanks Remy. I have wireshark and will try it. I will update the useragent. I did not try the rest libraries because I think you mentioned they use tidhttp anyway. Get back to you. – Bruce Long Nov 02 '14 at 02:08

0 Answers0