I have used code from here https://stackoverflow.com/a/16977731/2262863 and faced an issue when server returns 404 response. I have added to process url only if response have been 200 returned. But from what I see, code doesn`t continue with other urls. Here is an error I am getting:
System.AggregateException: One or more errors occurred. ---> System.AggregateException: One or more errors occurred. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
Tried to stop task, but got same issue.