0

I have a variable defined containing 1000 objects ($P) and also have a function that will take pipeline input and add a bunch of stuff to each object in the variable (Add-Data.AccountDetail). Instead of adding the stuff to each thing coming down the pipe one by one I was hoping that PoshRSJob would allow me to split the object being piped in and add stuff to it in parallel. IE take 5 lots of 200 and then process the adding to each in parallel.

I am not a natural coder but seem to get by, so I'm thankful for your patience and help if this is not clear.

$P = Get-PositionalData -AccountName T2000

$P | Start-RSJob -Name AccountDetail -Throttle 5 -ScriptBlock {
 $_ | Add-Data.AccountDetail

}

Get-RSJob | Receive-RSJob

I get the following, but nothing seems to be in the Receive-RSJob. I am certainly misunderstanding something fundamental here!

Error          : 
Verbose        : 
Debug          : 
Warning        : 
Progress       : 
HasMoreData    : True
HasErrors      : False
Output         : 
RunspacePoolID : fc217600-123f-4d73-9bcc-d1df6f87f6d8
Completed      : False
Batch          : fc217600-123f-4d73-9bcc-d1df6f87f6d8

Name           : AccountDetail
ID             : 169
State          : NotStarted
InputObject    : @{AccounId=T2000}
InstanceID     : 3ba5eeed-d48f-4712-8d32-08f5b12a46a3
Handle         : System.Management.Automation.PowerShellAsyncResult
Runspace       : 
InnerJob       : System.Management.Automation.PowerShell
Finished       : System.Threading.ManualResetEvent
Command        : 

                 $_| Add-Data.AccountDetail
HAL9256
  • 12,384
  • 1
  • 34
  • 46
Simon Johnson
  • 11
  • 1
  • 2

1 Answers1

0

I don't know what $P actually contains when it has 1000 objects, so I can only assume. You only need to tweak your Start-RSJob command job a bit.

$P | Start-RSJob -Name {$_.AccounId} -Throttle 5 -ScriptBlock {
 $_ | Add-Data.AccountDetail
} -FunctionsToLoad "Add-Data.AccountDetail"

The -Name parameter is the name of the job. You need to have a new name for each job that you want to run. I replaced your value with {$_.AccounId} because I'm assuming that each object in $P contains the property AccounId based on your example. You do not have to do it this way, but I'm just providing a way to add some uniqueness to each job. The nice thing about this parameter is that it knows the current pipeline object ($_).

You are using a custom function called Add-Data.AccountDetail. The scriptblock passed to Start-RSJob may not know anything about that function. You can use the FunctionsToLoad parameter to load your function. The syntax I have used assumes the function is already loaded in your current scope.

AdminOfThings
  • 23,946
  • 4
  • 17
  • 27
  • Appreciate that, thanks. When I stick a `Wait-RSJob` at the end then this is painfully slow. I may be putting too much on the function. My add-data.AccountDetail function builds a buffer of 100 objects and then makes a sql call on 100 at a time, once done there's a final object of 1000 which gets added to $P. `Get-RSJob | Where-Object {{State -like "Completed"} -and {HasMoreData -like "False"} -and {HasErrors -like "False"} } | Receive-RSJob ` is empty for a long time! – Simon Johnson Apr 17 '19 at 09:44
  • It could well be because I am doing lots of small jobs?? – Simon Johnson Apr 17 '19 at 10:05