0

I am using multithreading for getting Data for 7 Days,spawning 7 threads so that each thread will get data for a day.

Why multithreading? HttpResponseMessage can not hold more than 2GB of Data

I want to Display the total volume fetched by all the threads. How can I wait for the completion of all the threads?

class Program
    {
        static long totalVolume = 0;
        public static void CallToChildThread(String from, String to)
        {
            var httpclient = new HttpClient();
            String body = query;
            var webRequest = new HttpRequestMessage(HttpMethod.Post, api-url){Content = new StringContent(body, Encoding.UTF8, "application/json")};
            try
            {
                var response = httpclient.Send(webRequest);
                long size = (long)response.Content.Headers.ContentLength;
                totalVolume = totalVolume + size;
            }
            catch(Exception ex)
            {
                Console.WriteLine("Exception occured: for "+ from+"-"+to+" , " + ex.Message);
            }
        }
        static void Main(string[] args)
        {
            to = today;
            from = today - 1;
            count = 7;
            for (int i = 0; i <= count; i++)
            {
                Thread thread = new Thread(() => CallToChildThread(from, to));
                thread.Start(); 
                to = from;
                from = to-1;
            }

            //Code to wait for the completion of all the threads

            Console.WriteLine("total Volume fetched is "+totalVolume); 
         }
    }
Atul Takekar
  • 51
  • 13
  • Any particular reason to use `Thread` directly? – Guru Stron Nov 11 '21 at 21:16
  • `thread.join()` each one – ikegami Nov 11 '21 at 21:20
  • A web request is an inherently asynchronous operation, not a CPU bound operation no need to create multiple threads to do requests in parallel. – Servy Nov 11 '21 at 21:22
  • 1
    Yeah no need to use threads here. Each of these web client methods should have an async counterpart. Make the main method async, make the child call async, add all the tasks to a collection, then `await Task.WhenAll(tasks)`. – Patrick Tucci Nov 11 '21 at 21:23
  • AFAIK an `HttpResonse` *can* download more than 2GB, just use `ReadAsStreamAsync` and process it a bit at a time. You would also need to set `HttpCompletionOption.ResponseHeadersRead` – Charlieface Nov 11 '21 at 21:29
  • 1
    Your code (inside `Main`) is susceptible to this problem: [Captured variable in a loop in C#](https://stackoverflow.com/questions/271440/captured-variable-in-a-loop-in-c-sharp). – Theodor Zoulias Nov 12 '21 at 10:05
  • 1
    @TheodorZoulias - Thanks for the comment You were right. I was able to solve with Stackoverflow page linked in your comment – Atul Takekar Nov 17 '21 at 23:51

0 Answers0