How to make concurrent requests without creating multiple threads?

Question

Can someone please show how to make concurrent requests without creating multiple threads? E.g., I want a program that makes 100 web requests and I don't want more than 8 concurrent requests at any time. I don't want to create 8 threads for the 8 concurrent requests. When a thread makes an async request, the same thread can then be used to make the next request, and so on. I am sorry but I can't wrap my head around this, and would like to see the best solution out there. In case it wasn't clear, the requests I am talking about are async. I want to see a solution that does not use any locks, and uses the built-in classes to do the work.

This is some code I came up with but it does not do what it is supposed to do.

Task.Run(async () =>
                        {
                            var outstandingRequests = 0;
                            var requestCount = 0;
                            var tasks = new List<Task>(concurrentRequests);
                            while (requestCount < maxRequests)
                            {
                                if (outstandingRequests < concurrentRequests)
                                {
                                    tasks.Add(svc.GetDataAsync());  // a method that makes an async request
                                    Interlocked.Increment(ref outstandingRequests);
                                }
                                else
                                {
                                    var t = await Task.WhenAny(tasks);                                                                       
                                    Interlocked.Decrement(ref outstandingRequests);
                                    Interlocked.Increment(ref requestCount);
                                }
                            }
                            await Task.WhenAll(tasks);
                        }).Wait();

Output:

[] 1 Sending Request...Received Response 490,835.00 bytes in 15.6 sec
[] 2 Sending Request...
[] 3 Sending Request...
[] 4 Sending Request...
[] 5 Sending Request...
[] 6 Sending Request...
[] 7 Sending Request...
[] 8 Sending Request...
[] 9 Sending Request...

I have set concurrentRequests to 5, so there is some bug in above code as it is making 8 requests in parallel. Initially it made only 5 requests in parallel, but as soon as one request completed, it fired off 4 more requests (should have fired off only one more).

Had to fix some bugs, but it all works out now:

Task.Run(async () =>
                        {
                            var outstandingRequests = 0;
                            var requestCount = 0;
                            // adding and removing from a List<> at the same time is not thread-safe,
                            // so have to use a SynchronizedCollection<>
                            var tasks = new SynchronizedCollection<Task>();
                            while (requestCount < maxRequests)
                            {
                                if (outstandingRequests < concurrentRequests)
                                {
                                    tasks.Add(svc.GetDataAsync(uri)); // this will be your method that makes async web call and returns a Task to signal completion of async call
                                    Interlocked.Increment(ref outstandingRequests);
                                    Interlocked.Increment(ref requestCount);
                                }
                                else
                                {                                    
                                    **tasks.Remove(await Task.WhenAny(tasks));**
                                    Interlocked.Decrement(ref outstandingRequests);                                    
                                }
                            }
                            await Task.WhenAll(tasks);
                        }).Wait();

If there is a better way to do it, please let me know.

"without creating multiple threads" and "when a thread makes an async request, the same thread can be then used to make the next request" are conflicting statements. — Peter Ritchie, Mar 17 '13 at 03:03

score 4 · Answer 1 · answered Mar 17 '13 at 02:56

Looks like you are trying to reinvent the thread pool. Don't do that - just use existing functionality: http://msdn.microsoft.com/en-us/library/system.threading.threadpool.aspx
Or you can use async versions of request methods - they are based on the thread pool too.

score 1 · Answer 2 · edited May 23 '17 at 12:04

1

~~How about this:~~

Parallel.Invoke (new ParallelOptions { MaxDegreeOfParallelism = 8 },
    svcs.Select (svc => svc.GetDataAsync ()).ToArray ()) ;

There is a sample Microsoft implementation of a limited-concurrency task scheduler here. See SO questions System.Threading.Tasks - Limit the number of concurrent Tasks and .Net TPL: Limited Concurrency Level Task scheduler with task priority?.

edited May 23 '17 at 12:04

Community

1
1

answered Mar 17 '13 at 16:08

Anton Tykhyy

19,370
5
54
56

thanks. its useful to know. i have not tried it out. but will this not create 8 threads vs. single thread in my approach? is there some monitoring tool to know how many threads an app is creating? – morpheus Mar 17 '13 at 19:10
It shouldn't if your async code is written correctly, i.e. is not hogging threads. Your own code above is essentially a re-implementation of Parallel.Invoke. There are performance counters to monitor the number of both OS threads and managed threads in `.NET CLR LocksAndThreads' category. – Anton Tykhyy Mar 18 '13 at 00:16
OK, tried it and this does not work. You could try it yourself. What happens is that all the async calls will get fired all at once. Parallel.Invoke(new ParallelOptions { MaxDegreeOfParallelism = concurrentRequests }, () => { foreach (var i in Enumerable.Range(0, maxRequests)) { tasks.Add(MakeRequest(uri)); } }); Task.WaitAll(tasks.ToArray()); – morpheus Mar 18 '13 at 02:00
You *wrote* your code to fire all tasks at once! You are only feeding one action into Parallel.Invoke, small wonder it's not helping you any. However I was mistaken about Parallel.Invoke, it takes Actions not Tasks, which is not what you want. Please refer to the SO questions I cited in my answer. – Anton Tykhyy Mar 18 '13 at 02:25
you are correct. I modified the code so that I am feeding N actions into Parallel.Invoke. Each action is an async call. But the same thing happens. All N calls are made concurrently. So MaxDegreeOfParallelism is limiting the # of threads being created; it does not limit the # of Actions it will execute concurrently. Try it out. – morpheus Mar 18 '13 at 03:37
Yes, I've said above that Parallel.Invoke is not what you want. Use the solution from the first SO question referred to in my answer. – Anton Tykhyy Mar 18 '13 at 08:31

How to make concurrent requests without creating multiple threads?

2 Answers2