2

I would like to implement a -parallel Switch to one of my skripts

Non Parallel Version:

  $tmpArray | ForEach-Object {
          #region ... local Variables
          $stepWidthLocal = $stepWidth
<#
my code
#>

Parallel Funktion:

  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
          #region ... local Variables
          $stepWidthLocal = $using:stepWidth
<#
my code
#>

What I dont want is:

$myParallel = $true
if ($myParallel) {
  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
    #region ... local Variables
    $stepWidthLocal = $using:stepWidth
    <#
    my code
    #>
  } #end $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel
} #end if($myParallel) 
else {
  $tmpArray | ForEach-Object {
    #region ... local Variables
    $stepWidthLocal = $stepWidth
    <#
my code
#>
  } #end $tmpArray | ForEach-Object {
} #end else {

I want something like this:


$myCode = <#
define my Codeblock
#>
$myParallel = $true
if ($myParallel) {
  $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel {
    #region ... local Variables
    $stepWidthLocal = $using:stepWidth
    $myCode
  } #end $tmpArray | ForEach-Object -ThrottleLimit ($parallelBlocks * 10) -Parallel
} #end if($myParallel) 
else {
  $tmpArray | ForEach-Object {
    #region ... local Variables
    $stepWidthLocal = $stepWidth
    $myCode
  } #end $tmpArray | ForEach-Object {
} #end else {

Now I want to create some kind of switch statement without duplicating the entire code (block <# my code#>).

is this possible?

mklement0
  • 382,024
  • 64
  • 607
  • 775
Farbkreis
  • 604
  • 3
  • 12

2 Answers2

1
$arguments = @{ }
if ($myParallel) {
    $arguments = @{ parallel = $true; $throttlelimit = 10 }
}
$tmpArray | ForEach-Object @arguments {
...
}
David Jones
  • 2,879
  • 2
  • 18
  • 23
  • I tested this solution but I failed as I cannot pass the variables to the block. You need to pass the variables with $using: to the parallelblock: https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/ which doesnt work in the nonparellel block. So you could implement a $useParallel switch but as you cannot pass the $useParallel itself, this doesnt work either. So for all I know, the idea is great, but it doesnt work – Farbkreis Sep 02 '21 at 05:47
1

You can define the reusable code as a script block, but note that you won't be able to use it directly in your ForEach-Object -Parallel script block and instead have to recreate it there, via its string representation passed to the static [scriptblock]::Create() method; using a simplified example:

# Your reusable code block.
# Note the .ToString() call to obtain its string representation.
$myCodeSource = {
  "hi from thread $([System.Threading.Thread]::CurrentThread.ManagedThreadId)"
}.ToString()


1, 2 | ForEach-Object -Parallel { 
  # ...
  # Note: You can pass arguments, if the script block is prepared to handle them.
  & ([scriptblock]::Create($using:myCodeSource)) 
}

Note: This answer contains an analogous solution for using a function from the caller's scope in a ForEach-Object -Parallel script block.

The above output something like the following:

hi from thread 35
hi from thread 36

Note (as of PowerShell 7.2):

  • ForEach-Object -Parallel actively prevents direct use of script blocks from the caller's scope (accessed via the $using:) scope, because using script blocks across threads can lead to thread-safety issues; curiously, however, the related Start-ThreadJob does accept script blocks via $using: - though that may have been an oversight.

    • Recreating a script block via its string representation, as shown above, works around this limitation.
  • GitHub issue #12378 discusses this behavior, including a possible enhancement to let ForEach-Object itself automatically recreate the script block in a thread-safe manner.

    • More generally, GitHub issue #12240 proposes an opt-in mechanism that would allow copying the caller's definitions to each thread, making $using: references unnecessary.
mklement0
  • 382,024
  • 64
  • 607
  • 775