1

I have a script which I use to load a webpage, retrieve information from said webpage, then output said information to a file. It had been working perfectly until today, when I have been getting an error which reads:

invoke-webrequest : Response object error 'ASP 0251 : 80004005' 
Response Buffer Limit Exceeded 
/foo/Reports/SearchLocation.asp, line 0 
Execution of the ASP page caused the Response Buffer to exceed its configured limit. 
At C:\path.ps1:7 char:12
+     $url = invoke-webrequest "http://url/ ...
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

I do not believe there have been any changes to the site which it is pulling its data from, and the file it is getting the input information from has no formatting errors.

Some googling around leads me to believe that the issue is that the page has more than 4 mb worth of data to load, and the default buffer size is 4 mb, but I can't find any instructions for how to change the buffer size in PowerShell.

I came across the clear-webconfiguration cmdlet, but I'm not certain whether or not that is what I need, or how exactly to implement it within my script. Here is the main portion of my code:

foreach($c in $csv){
    [array]$tags = $null
    $url = invoke-webrequest "http://url.com" -UseDefaultCredentials
    $table = $url.ParsedHTML.getElementsByTagName('table')[7]
    $rows = $table.getElementsByTagName('tr')
    if($c.'Was this user on the old file?' -eq 'Yes'){
        if($table.innerText -like "*N/A*" ){
            $man = $c.'Manager Name' -replace ',',';'
            $badusers += $c.'User ID' + "," + $c.Name + "," + $man + "," + $c.'CC'
        }
        else{
            foreach($row in $rows){
                $bcol = $row.getElementsByTagName('td') | Where-Object{$_.cellIndex -eq 1} | select -First 1 
                $ccol = $row.getElementsByTagName('td') | Where-Object{$_.cellIndex -eq 7} | select -First 1
                $bcol = $bcol.innerText
                $ccol = $ccol.innerText
                if($ccol -ne $c.'CC'){
                    $tags += $bcol + ",," + $c.'CC' + "," + $c.'User ID'
                }
            }
            if($tags -ne $null){
                $results += $tags
            }
        }
    }
}

Any help on solving this issue is much appreciated.

Cameron
  • 171
  • 3
  • 15
  • To investigate, use `(Invoke-WebRequest $url -Method Head).Headers.'Content-Length'`: you should be able to get the file size without downloading it. – sodawillow Sep 13 '17 at 17:48
  • @sodawillow, I tried doing a simple loop to check the content length for each page, but I am receiving a (500) internal server error each time. This is very strange because I have several different scripts which draw from the same resource, and none are having this issue except this one (I have run each of them since this issue appeared). I have checked to ensure there are no typos in the url and there are not. I'm at a loss as for why it would do this – Cameron Sep 13 '17 at 19:45
  • Check this Q: [Detailed 500 error message, ASP + IIS 7.5](https://stackoverflow.com/questions/2640526/detailed-500-error-message-asp-iis-7-5): you can maybe get some useful debugging info this way. Funnily, I visited it a few hours earlier today :). – sodawillow Sep 13 '17 at 19:48

0 Answers0