2

i received this error on PowerShell 7:

Total Urls to process: 12496
i: 1 | total:
RuntimeException:
Line |
  45 |      $percent = [int](100 * $i / $lines)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Attempted to divide by zero.

development this script:

$srcfile = "C:\Users\wnune\OneDrive\Escritorio\imagenes\cardlist.txt"
$urls = Get-Content $srcfile
$lines = 0
switch -File $srcfile { default { ++$lines } }
Write-Host "Total Urls to process: $lines "
$i = 0
Write-Progress -Activity "Downloading files" -Status "In progress" -PercentComplete $i;
$urls | ForEach-Object -Parallel {
    $url = $_
    try {
        $filename = Split-Path $url -Leaf
        $destination = "C:\Users\wnune\OneDrive\Escritorio\imagenes\$filename"
        $ProgressPreference = 'SilentlyContinue'
        $response = Invoke-WebRequest -Uri $url -ErrorAction SilentlyContinue
        if ($response.StatusCode -ne 200) {
            Write-Warning "============================================="
            Write-Warning "Url $url return Error. "
            continue
        }
        if (Test-Path $destination) {
            Write-Warning "============================================="
            Write-Warning "File Exist in Destination: $filename "
            continue
        }
        $job = Start-BitsTransfer -Source $url -Destination $destination -Asynchronous
        while (($job | Get-BitsTransfer).JobState -eq "Transferring" -or ($job | Get-BitsTransfer).JobState -eq "Connecting")
        {
            Start-Sleep -m 250
        }
        Switch(($job | Get-BitsTransfer).JobState)
        {
            "Transferred" {
                Complete-BitsTransfer -BitsJob $job
            }
            "Error" {
                $job | Format-List
            }
        }
    }
    catch 
    {
        Write-Warning "============================================="
        Write-Warning "There was an error Downloading"
        Write-Warning "url:         $url"
        Write-Warning "file:        $filename"
        Write-Warning "Exception Message:"
        Write-Warning "$($_.Exception.Message)"
    }
    
    $i++
    Write-Host "i: $i | total: $lines"
    $percent = [int](100 * $i / $lines)
    Write-Progress -Activity "Downloading files" -Status "In progress" -PercentComplete $percent
}
Write-Progress -Activity "Downloading files" -Status "Completed" -Completed

Assuming this is because I'm implementing -Parallel try using a sync object:

$syncObject = [System.Object]::new()
$srcfile = "C:\Users\wnune\OneDrive\Escritorio\imagenes\cardlist.txt"
$urls = Get-Content $srcfile
$lines = 0
switch -File $srcfile { default { ++$lines } }
Write-Host "Total Urls to process: $lines "
$i = 0
Write-Progress -Activity "Downloading files" -Status "In progress" -PercentComplete $i;
$urls | ForEach-Object -Parallel {
    [System.Threading.Monitor]::Enter($syncObject)
    try {
        $url = $_
        $filename = Split-Path $url -Leaf
        $destination = "C:\Users\wnune\OneDrive\Escritorio\imagenes\$filename"
        $ProgressPreference = 'SilentlyContinue'
        $response = Invoke-WebRequest -Uri $url -ErrorAction SilentlyContinue
        if ($response.StatusCode -ne 200) {
            Write-Warning "============================================="
            Write-Warning "Url $url return Error. "
            continue
        }
        if (Test-Path $destination) {
            Write-Warning "============================================="
            Write-Warning "File Exist in Destination: $filename "
            continue
        }
        $job = Start-BitsTransfer -Source $url -Destination $destination -Asynchronous
        while (($job | Get-BitsTransfer).JobState -eq "Transferring" -or ($job | Get-BitsTransfer).JobState -eq "Connecting")
        {
            Start-Sleep -m 250
        }
        Switch(($job | Get-BitsTransfer).JobState)
        {
            "Transferred" {
                Complete-BitsTransfer -BitsJob $job
            }
            "Error" {
                $job | Format-List
            }
        }
    }
    catch 
    {
        Write-Warning "============================================="
        Write-Warning "There was an error Downloading"
        Write-Warning "url:         $url"
        Write-Warning "file:        $filename"
        Write-Warning "Exception Message:"
        Write-Warning "$($_.Exception.Message)"
    }
    finally 
    {
        $i++
        Write-Host "i: $i | total: $lines"
        $percent = [int](100 * $i / $lines)
        Write-Progress -Activity "Downloading files" -Status "In progress" -PercentComplete $percent
        [System.Threading.Monitor]::Exit($syncObject)
    }
}
Write-Progress -Activity "Downloading files" -Status "Completed" -Completed

get another error:

Total Urls to process: 12496
MethodInvocationException:
Line |
   2 |      [System.Threading.Monitor]::Enter($syncObject)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Exception calling "Enter" with "1" argument(s): "Value cannot be null."

I have no idea what is wrong or why this happens, if I am respecting the order of execution and I am creating the synchronization object from the beginning; Can someone help me with this???

the main idea is to process the 12496 Urls... and create a progress bar based on the number of Urls processed in parallel.

mklement0
  • 382,024
  • 64
  • 607
  • 775

1 Answers1

2

The PowerShell v7+ ForEach-Object -Parallel feature uses separate, thread-based runspaces to execute code in parallel.

As in any PowerShell code (script block, { ... }) that executes out of runspace, you must use the $using: scope in order to refer to variable values in the caller's scope.

  • This answer provides an overview of all contexts in which $using: is required.

A simple example:

$i = 42
1..3 |
  ForEach-Object -Parallel {
    [pscustomobject] @{
      'Thread-local $i' = $i
      '$using:i value' = $using:i
    }
  }  

Output:

Thread-local $i $using:i value
--------------- --------------
                            42
                            42
                            42

As you can see, the $i variable has no value, because it refers to a thread-local (runspace-local) variable that hasn't been initialized.


Updating values in the caller's scope:

$using: references are only ever the values of variables from the caller's scopes, not the variables themselves.

If values happen to be instance of .NET reference types, you can (potentially) update them, by setting their properties and/or calling their methods, but this does not work for values that happen to be instances of value types, such as [int].

In order to update instances of the latter, wrap them in a reference type, such as a hashtable.

In order to make updating a value thread-safe, explicit synchronization is required, such as via System.Threading.Monitor:Thanks, Santiago Squarzon.

$iWrapper = @{ Value = 42 }
1..3 |
  ForEach-Object -Parallel {
    # Lock the hashtable so that no other thread can update it.
    [System.Threading.Monitor]::Enter($using:iWrapper)
      # Update its (one and only) entry.
      (++($using:iWrapper).Value)
    # Release the lock.
    [System.Threading.Monitor]::Exit($using:iWrapper)
  }  

Output:

43
44
45
mklement0
  • 382,024
  • 64
  • 607
  • 775
  • why the other answer, not have an implement example like this? –  Dec 24 '22 at 00:43
  • i havce try to implement it and get error when y try to: `$using:i++` –  Dec 24 '22 at 02:46
  • @ArcanisGK507, `$using:` references are variable _values_, not _variables_, so you cannot update _variables_, but you can act on instances of .NET _reference types_ (and then you may need cross-thread synchronization, as you've attempted). – mklement0 Dec 24 '22 at 02:54
  • @ArcanisGK507, as for the other answer: it attempts to provide a _general overview_ of all contexts where `$using:` is required. By linking to the relevant documentation, the rules for concrete usage of `$using:` can be _inferred_. – mklement0 Dec 24 '22 at 02:56
  • so nothing I've seen works... the idea is to count the urls iterated in each thread in parallel... hahahahaha this seems like a joke... I assume that the synchronization object that is in my second example script is the path correct and not implement $using: scope as you suggested –  Dec 24 '22 at 02:57
  • i will try with $global:i ... i read about it .. but progress bar is out of scope i think XD –  Dec 24 '22 at 03:01
  • @SantiagoSquarzon i use [System.Threading.Monitor] and get error ... XD –  Dec 24 '22 at 03:30
  • At the moment i have other error; use [hashtable] is working first time in re-run i get ***Exception setting "CursorPosition": "A command that prompts the user failed because the host program or the command type does not support user interaction. Try a host program that supports user interaction, such as the PowerShell Console, and remove prompt-related commands from command types that do not support user interaction."*** –  Dec 24 '22 at 03:34
  • @SantiagoSquarzon but if i whant to increace the value of a property inside $syncObject like a counter??? –  Dec 24 '22 at 03:42
  • chek my wife have create a question: https://stackoverflow.com/q/74905498/20850232 becouse i cant do that ... so SO show say that my question is low quality lol ... –  Dec 24 '22 at 03:55