Making Requests in Parallel with Powershell results in "You cannot call a method on a null-valued expression"

Question

I am trying to use powershell to make webrequests in parallel. I have a function "MakeRequest" that makes the type of request I need to make. This function works fine when run one-off, or when run in sequence. However, when I attempt to use either Start-Job or define a workflow to execute requests in parallel, I run into the error "You cannot call a method on a null-valued expression". The error is thrown on when i attempt to get the response from the web request "$response = $webRequest.GetResponse()"

What am i doing wrong / missing?

[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

$MakeRequest = {
    param($machineName, [int]$port)
    Write-Host $machineName $port
    $url = "https://$machineName" + ":$port/path"
    $webRequest = [net.WebRequest]::Create($url)
    $webRequest.Headers.Add('MyHeader', 'MyValue')
    # 10 Minute Timeout
    $webRequest.Timeout = 600000

    $returnCode = 1
    $output = ""
    Try {
        Write-Host $webRequest
        $response = $webRequest.GetResponse()
        $statusCode = [int]$response.StatusCode

        if ($statusCode -ne 200) {
            Write-Host "Server $machineName did not return a 200, and instead returned a $statusCode"
            Exit 1
        }

        [IO.Stream] $stream = $response.GetResponseStream()
        [IO.StreamReader] $reader = New-Object IO.StreamReader($stream)
        [string] $output = $reader.readToEnd()
        $stream.flush()
        $stream.close()
        $returnCode = 0
    } Catch [system.exception] {
        [IO.Stream] $stream = $_.Exception.Response.GetResponseStream()
        [IO.StreamReader] $reader = New-Object IO.StreamReader($stream)
        [string] $output = $reader.readToEnd()
        $stream.flush()
        $stream.close()
    } Finally {
        Write-Debug "$machineName $output"
    }
}

@("mach1, mach2, mach3" ) | ForEach-Object { Start-Job -ScriptBlock $MakeRequest -ArgumentList $_, 10443 }

Get-Job | Wait-Job

Get-Job | % { Receive-Job $_.Id; Remove-Job $_.Id }

Have you looked at using the `Invoke-WebRequest` https://technet.microsoft.com/en-us/library/hh849901.aspx method instead? — Eris, Jun 28 '15 at 22:44
I tried that an also get an error - "The underlying connection was closed: An unexpected error occurred on a send. + CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebExc eption + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand + PSComputerName : localhost" — badazzhindu, Jun 29 '15 at 12:35
Try to put `[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}` inside of `$MakeRequest` script block. — user4003407, Jun 30 '15 at 19:15

score 2 · Answer 1 · edited Mar 08 '23 at 15:16

Assuming that the code you have posted is the same code you are using I believe that the issue is the way in which you are passing your arguments to the scriptblock. Am I correct in assuming that you want the foreach-object to execute 3 times (once for mach1, once for mach2, once for mach3)?

If this assumption is correct then the problem is that you are piping only the string "mach1, mach2, mach3" to the foreach, not an array of 3 strings. The loop executes once and passes "mach1, mach2, mach3" as the first argument.

I made a script similar to yours to test it out and it works if the strings are piped like this:

@("mach1","mach2","mach3") | Foreach-Object {your start-job code here}

Note that each item is now enclosed in its own quotes.

Making Requests in Parallel with Powershell results in "You cannot call a method on a null-valued expression"

1 Answers1

Linked