C# Task Parallel Library first time slow

Question

Trying to perform a number of HTTP Get requests in parallel, one per task. If I do the Gets through internet explorer they return almost instantaneously, but when calling in code through tasks, the first time I fire them off they take a good few seconds to return, but running a second time they return as I would expect. So either I'm doing something which is blocking something, or for some reason the threads are not starting? It's my first time trying to use TPL.

Here's the basic lookup class:

public class Lookup
{
  public string Name { get; set; }
  public string URL { get; set; }

  public Lookup(string Name, string URL)
  {
    this.Name = Name;
    this.URL = URL;
  }

  public LookupReturn DoLookup()
  {
    LookupReturn d = new LookupReturn();
    d.Name = this.Name;
    d.URL = this.URL;

    WebRequest wrGETURL;
    wrGETURL = WebRequest.Create(this.URL);

    Stream objStream;
    objStream = wrGETURL.GetResponse().GetResponseStream();

    StreamReader objReader = new StreamReader(objStream);

    string sLine = objReader.ReadToEnd();
    d.Result = sLine;
    return d;
  }
}

And the return type is simply:

public class LookupReturn
{
  public string Name { get; set; }
  public string Result { get; set; }
  public string URL { get; set; }
}

So the attempt to run this in parallel - am testing from a Winforms GUI but eventually will be in a WCF service.

public partial class Form1 : Form
{
  private List<Lookup> Lookups = new List<Lookup>();

  private async void btnRunLookups_Click(object sender, EventArgs e)
  {
    Lookups.Add(new Lookup("Name1", "http://geturl1 "));
    Lookups.Add(new Lookup("Name2", "http://geturl2 ")); 
    // More lookups…  

    int workerThreads, complete;
    ThreadPool.GetMinThreads(out workerThreads, out complete);

    ThreadPool.SetMinThreads(100, complete);

    btnRunLookups.Visible = false;
    List <Task<LookupReturn>> lookupTasks = new List<Task<LookupReturn>>();
    foreach(Lookup dl in Lookups)
    {
      lbLookups.Items.Add("Starting task for " + dl.URL);

      Task<LookupReturn> t = new Task<LookupReturn>(() => dl.DoLookup() );
      lookupTasks.Add(t);
      t.Start();
    }

    //await Task.WhenAny(
    //  Task.WhenAll(lookupTasks.ToArray<Task<LookupReturn>>()), 
    //  Task.Delay(3000)
    //  );

    // This takes a good few seconds the first time
    await Task.WhenAll(lookupTasks.ToArray<Task<LookupReturn>>());

    // Now I need to see which ones completed and returned a result
    foreach (var x in lookupTasks)
    {
      if (x.IsCompleted)
      {
        lbLookups.Items.Add(x.Result);
      }
      else
      {
        // lbLookups.Items.Add("Not finished " + x.Result.Name);
      }
    }
    btnRunLookups.Visible = true;
  }

What are you doing with this line `ThreadPool.SetMinThreads(100, complete);` ? From msdn "You can use the SetMinThreads method to increase the minimum number of threads. However, unnecessarily increasing these values can cause performance problems. **If too many tasks start at the same time, all of them might appear to be slow**. In most cases, the thread pool will perform better with its own algorithm for allocating threads. Reducing the minimum to less than the number of processors can also hurt performance." — Sriram Sakthivel, Mar 24 '15 at 15:44
When you say that it takes longer the first time than the second, are you running the program two separate times? Or do you start the program and execute your lookup code twice? — Brian, Mar 24 '15 at 15:48
SetMinThreads I had read somewhere else was helpful if you needed more threads than you have cores - it makes no difference if it is taken out. — Ray Browning, Mar 24 '15 at 16:18

score 5 · Accepted Answer · edited May 23 '17 at 11:53

5

Other people have noted that HttpWebRequest can take a long time on its first request because it's looking for proxy information. Try setting its Proxy property to null.

wrGETURL = WebRequest.Create(this.URL);
wrGETURL.Proxy = null;

edited May 23 '17 at 11:53

Community

1
1

answered Mar 24 '15 at 15:50

StriplingWarrior

151,543
27
246
315

score 1 · Answer 2 · answered Mar 24 '15 at 15:50

1

Most likely the problem is that the program is doing some first-time setup stuff (DNS resolution, proxy detection, etc.) on the first call to GetResponse. The proxy detection, in particular, can take a good long while.

answered Mar 24 '15 at 15:50

Jim Mischel

131,090
20
188
351

C# Task Parallel Library first time slow

2 Answers2