HttpWebRequest "operation has timed out"

Question

I am using an httpWebRequest in my windows app to download files from a webserver (sURL), to a local folder (fileDestination) as such:

Public Function DownloadFile(ByVal sURL As String, _
                        ByVal fileDestination As String, _
                        ByVal WebRequestType As String) As Boolean  


    Dim URLReq As HttpWebRequest
        Dim URLRes As HttpWebResponse
        Dim FileStreamer As FileStream
        Dim bBuffer(999) As Byte
        Dim iBytesRead As Integer
        Dim folderDestination As String
        Dim sChunks As Stream

        Try

            FileStreamer = New FileStream(fileDestination, FileMode.Create)

            URLReq = WebRequest.Create(sURL)
            URLRes = URLReq.GetResponse 'Error occurs here!!
            sChunks = URLReq.GetResponse.GetResponseStream
            DownloadProgressBar.Value = 0
            DownloadProgressBar.Maximum = URLRes.ContentLength

            Do
                iBytesRead = sChunks.Read(bBuffer, 0, 1000)
                FileStreamer.Write(bBuffer, 0, iBytesRead)
            Loop Until iBytesRead = 0

            sChunks.Close()
            FileStreamer.Close()
            URLRes.Close()

            Return True
        Catch ex As Exception
            Return False
        End Try

End Function

This works fine for the first few files. But then it starts giving the following error on the URLReq.GetResponse line:

"operation has timed out"

Anyone know what could be causing this?

I noticed you tagged the question as asp.net, is this all occurring in an asp.net web page or is this a windows app that is request from an asp.net website? — OneSHOT, May 26 '11 at 14:14
@Urbycoz, just to ask the obvious, are you able to download whatever you're trying to access using a regular web browser just fine? Also, is whatever you are trying to download from a server that you control or from a third-party? If the latter, some sites (Google in particular) inspect the user agent header looking for bots. Try manually setting the request's user agent to some common user agent. — Chris Haas, May 26 '11 at 14:22
@Chris. Yes, I can download all of the files directly from a browser. Whatsmore it is not any one particular file that always fails. If I do them in a different order, it fails on different files. No, not downloading from third party- it's our own server. — Urbycoz, May 26 '11 at 14:46
@OneShot. Oops, just realised my mistake. It's actually a windows app downloaded using clickOnce technology. No "asp.net" involved. I've removed the tag. +1 for pointing that out. — Urbycoz, May 26 '11 at 14:49
You're not by any chance targeting the 1.1 framework, are you? — Chris Haas, May 26 '11 at 15:05
@Urbycoz, Check this post out, he says he needed to add a `Request.Abort` call. http://www.andrewdothay.net/blog/2007/07/25/SystemNetWebExceptionTheOperationHasTimedout.aspx — Chris Haas, May 26 '11 at 15:13
mmm...Looks like I may have too many requests running at once. Seems 4 is the maximum. — Urbycoz, May 26 '11 at 15:32
@Urbycoz, If that's the case you might be able to set `System.Net.ServicePointManager.DefaultConnectionLimit` — Chris Haas, May 26 '11 at 16:26
@Chris. You are dead right! Thank you. I set System.Net.ServicePointManager.DefaultConnectionLimit=10, and it fixed the problem. It was actually set to 2 originally. If you post this as an answer, I'll accept it. — Urbycoz, May 27 '11 at 07:38

score 2 · Accepted Answer · edited Aug 22 '12 at 19:18

2

If you're targeting the 1.1 framework and are running multiple concurrent requests try setting System.Net.ServicePointManager.DefaultConnectionLimit

edited Aug 22 '12 at 19:18

Brian Webster

30,033
48
152
225

answered May 27 '11 at 12:45

Chris Haas

53,986
12
141
274

score 1 · Answer 2 · answered May 26 '11 at 13:47

1

The timeout is set to 10000 milliseconds (or 10 seconds). Is that long enough for the roundtrip to the web server?

The MSDN Documentation says the default is 100 seconds (100000 ms) is there any reason you changed that default?

answered May 26 '11 at 13:47

Colin Mackay

18,736
7
61
88

Yes. It was taking forever to bomb out every time, and I got sick of waiting. I'll take it back out again. Problem still occurs. – Urbycoz May 26 '11 at 13:54

HttpWebRequest "operation has timed out"

2 Answers2

Linked