Using Await in BackgroundWorker causes WorkerComplete event to be raised

Question

I have this weird problem. I need to invoke a process from within a background worker

Private Shared _process As Process
Private Shared _StartInfo As ProcessStartInfo
Private WithEvents _bwConvertMedia As New BackgroundWorker

Here is the work in DoWorkAsync

Private Async Sub _bwConvertMedia_DoWorkAsync(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles _bwConvertMedia.DoWork
  For AI = 1 To 100
    _StartInfo = New ProcessStartInfo(".\mycmd.exe", "-1")
    _StartInfo.RedirectStandardOutput = True
    _StartInfo.UseShellExecute = False
    _StartInfo.CreateNoWindow = True
    _StartInfo.RedirectStandardError = True

    _process = New Process() With {.EnableRaisingEvents = True, .StartInfo = _StartInfo}
    AddHandler _process.OutputDataReceived, AddressOf OutputHandler
    AddHandler _process.ErrorDataReceived, AddressOf ErrorHandler
    AddHandler _process.Exited, AddressOf Exited
    Try
      aSuccess = Await AwaitProcess()
    Catch ex As Exception
    End Try
    _bwConvertMedia.ReportProgress(ai)
  Next

And here the

Private Shared Async Function AwaitProcess() As Task(Of Integer)
  _tcs = New TaskCompletionSource(Of Integer)
  _status.Converting = True
  _Error.Clear()
  _process.Start()
  _process.BeginErrorReadLine()
  _process.BeginOutputReadLine()    
  Return Await _tcs.Task
End Function

The issue is that when the Await _tcs.Task is executed the _bwConvertMedia RunWorkerCompleted procedure is executed so when I do call the _bwConvertMedia.ReportProgress(ai)

I get an error that the worker is already finished.

Why is that? can you help me?

What happens is

DoWork - iteration 1
at wait process 1
RunWorkerComplete
DoWork iteration 2-100

The correct behavior is that the background worker invokes 100 times the process and THEN it finishes the execution and calls the RunWorkerCompleted

My suggestion is to use `Task.Factory()` if those tasks can be run in parallel, `Task.Run()` if you can manage the eventual concurrency (I don't know what those processes do). When you schedule a task, in any way, you can evaluate it using `ContinueWith()` to test the status and elaborate it if `.Status = TaskStatus.RanToCompletion`. — Jimi, Jan 03 '18 at 00:44
The processes are encoding processes so I do not have advantage to have them run in parallel as they use 100% of the GPU/CPU to decode/encode video. If I lunch 2 of them they will simply use 50% each and double the time :). Jimi, thank you for your suggestion, do you have an example? — Massimo Savazzi, Jan 03 '18 at 11:47
This is a special case, since you apparently don't directly manage the process of encoding and is not clear whether this process is concurrently CPU/GPU and IO bound (encodes in memory first, then writes to storage or both at the same time). It looks like you are using a Background Worker just to keep the UI responsive while trying to serialize the process output. But, In this scenario, I think that scheduling a number of parallel processes = to the hardware (CPU) physical threads could be an idea. — Jimi, Jan 03 '18 at 12:33
I wrote [something like this](https://stackoverflow.com/questions/47913838/keep-ui-thread-responsive-when-running-long-task-in-windows-forms/47916647#47916647) a few days ago and something else related to [starting a process and waiting its ending](https://stackoverflow.com/questions/47836230/positioning-the-windows-system-date-time-clock-window/47843749#47843749). Those two combined already give you an answer. They are, however, written in C#. Take a look and let me know. — Jimi, Jan 03 '18 at 12:37
I put together a couple of examples, in case you you still need it. — Jimi, Jan 03 '18 at 22:07

Jimi · Accepted Answer · 2018-01-04T17:05:12.533

I made some modifications to the code I previously linked and here are two examples of a sequential non-blocking Async/Await procedure and a non-blocking Parallel procedure using Task.Factory.

Since I can't test your program, I simply used Tracert.exe to simulate a stdout result to update the User Interface.

To synchronize the running tasks/threads with the UI, I used in the first case the .SynchronizingObject of the processes and in the second the TaskScheduler method TaskScheduler.FromCurrentSynchronizationContext().

The output from Tracert.exe is passed to two TextBoxes. In the parallel example, I inserted a delay of 1 second between Tasks, to see how the two TextBoxes are updated.

The Async/Await example can be modified to work differently, since you don't need to wait for a task to complete to start another.

The ProcessStartInfo and Process objects are added to a Pool using a List(Of ProcessStartInfo) and a List(Of Process).

These are used in both examples. Define a correct scope.

Public psInfoPool As List(Of ProcessStartInfo)
Public ProcessPool As List(Of Process)

Sequential Async/Await

The delegate is used with SynchronizingObject.BeginInvoke if InvokeRequired = true

Public Delegate Sub UpdUI(_object As TextBox, _value As String)

Public Sub UpdateUIDelegate(control As TextBox, _input As String)
    control.AppendText(_input)
End Sub

    Dim NumberOfProcesses As Integer
    For x = 0 To 1
        Dim OutCtl As TextBox = If(x = 0, Me.TextBox1, Me.TextBox2)
        Dim _result As Integer = Await Task.Run(Async Function() As Task(Of Integer)
                                     Return Await Test_SequentialAsync("192.168.1.1", OutCtl)
                                 End Function)
        NumberOfProcesses += _result
    Next

The MediaToConvert parameter would be the name of the file to convert when you adapt the examples to your needs. The OutCtl parameter is just the TextBox used for the output

Public Async Function Test_SequentialAsync(ByVal MediaToConvert As String, OutCtl As TextBox) As Task(Of Integer)
    Dim _CurrentProcessInfo As Integer
    Dim _CurrentProcess As Integer

    Dim ExitCode As Integer = Await Task.Run(Function() As Integer

        Dim _processexitcode As Integer

        psInfoPool.Add(New ProcessStartInfo)
        _CurrentProcessInfo = psInfoPool.Count - 1
        psInfoPool(_CurrentProcessInfo).RedirectStandardOutput = True
        psInfoPool(_CurrentProcessInfo).CreateNoWindow = True
        psInfoPool(_CurrentProcessInfo).UseShellExecute = False
        'Name of the executable to start
        psInfoPool(_CurrentProcessInfo).FileName = "Tracert"    'psInfo.FileName = ".\mycmd.exe"""
        'Parameter(s) to pass to the executable
        psInfoPool(_CurrentProcessInfo).Arguments = MediaToConvert
        psInfoPool(_CurrentProcessInfo).WindowStyle = ProcessWindowStyle.Hidden

        ProcessPool.Add(New Process)
        _CurrentProcess = ProcessPool.Count - 1

        ProcessPool(_CurrentProcess) = New Process() With {.StartInfo = psInfoPool(_CurrentProcessInfo),
                                                           .EnableRaisingEvents = True,
                                                           .SynchronizingObject = Me}

        ProcessPool(_CurrentProcess).Start()
        ProcessPool(_CurrentProcess).BeginOutputReadLine()
        AddHandler ProcessPool(_CurrentProcess).OutputDataReceived,
            Sub(sender As Object, e As DataReceivedEventArgs)
                    If e.Data IsNot Nothing Then
                        If ProcessPool(_CurrentProcess).SynchronizingObject.InvokeRequired Then
                            ProcessPool(_CurrentProcess).SynchronizingObject.BeginInvoke(
                                                         New UpdUI(AddressOf UpdateUIDelegate),
                                                         New Object() {OutCtl,
                                                         e.Data + Environment.NewLine})
                        Else
                            OutCtl.AppendText(e.Data + Environment.NewLine)
                        End If
                    End If
            End Sub

        'Add an event handler for the Exited event
        AddHandler ProcessPool(_CurrentProcess).Exited,
                Sub(source As Object, ev As EventArgs)
                    _processexitcode = ProcessPool(_CurrentProcess).ExitCode
                    Console.WriteLine("The process has exited. Code: {0}  Time: {1}",
                    _processexitcode,
                    ProcessPool(_CurrentProcess).ExitTime)
                End Sub

        ProcessPool(_CurrentProcess).WaitForExit()
        ProcessPool(_CurrentProcess).Close()
        Return _processexitcode
    End Function)

    Return If(ExitCode = 0, 1, 0)

End Function

Parallel processes using Task.Fatory

Define a Scheduler and associate it with the current context

Public _Scheduler As TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext()

To use Await Task.Delay(1000) you must be in an Async method, but it's just for testing the output, it's not needed.

For x = 0 To 1
    Dim OutCtl As TextBox = If(x = 0, Me.TextBox1, Me.TextBox2)
    Dim _result As Integer = Test_ParallelTasks("192.168.1.1", OutCtl)
    Await Task.Delay(1000)
    NumberOfProcesses += _result
Next

Note that a new Task is created when the OutputDataReceived event handler reports that new data has been received. The UI is updated accordingly using DataReceivedEventArgs e.Data.

Private Function Test_ParallelTasks(ByVal MediaToConvert As String, OutCtl As TextBox) As Integer
    Dim _processexitcode As Integer
    Dim _CurrentProcessInfo As Integer
    Dim _CurrentProcess As Integer

    Task.Factory.StartNew(Function()
        psInfoPool.Add(New ProcessStartInfo)
        _CurrentProcessInfo = psInfoPool.Count - 1
        psInfoPool(_CurrentProcessInfo).RedirectStandardOutput = True
        psInfoPool(_CurrentProcessInfo).CreateNoWindow = True
        psInfoPool(_CurrentProcessInfo).UseShellExecute = False
        psInfoPool(_CurrentProcessInfo).FileName = "Tracert"  'psInfo.FileName = ".\mycmd.exe"
        psInfoPool(_CurrentProcessInfo).Arguments = MediaToConvert
        psInfoPool(_CurrentProcessInfo).WindowStyle = ProcessWindowStyle.Hidden

        ProcessPool.Add(New Process)
        _CurrentProcess = ProcessPool.Count - 1
        ProcessPool(_CurrentProcess) = New Process() With {.StartInfo = psInfoPool(_CurrentProcessInfo),
                                                           .EnableRaisingEvents = True,
                                                           .SynchronizingObject = Me}

        ProcessPool(_CurrentProcess).Start()
        ProcessPool(_CurrentProcess).BeginOutputReadLine()

        AddHandler ProcessPool(_CurrentProcess).OutputDataReceived,
            Sub(sender As Object, e As DataReceivedEventArgs)
                If e.Data IsNot Nothing Then
                    Try
                        'Update the UI or report progress 
                        Dim UpdateUI As Task = Task.Factory.StartNew(Sub()
                        Try
                            OutCtl.AppendText(e.Data + Environment.NewLine)
                        Catch exp As Exception
                              'An exception may raise if the form is closed
                        End Try

                        End Sub, CancellationToken.None, TaskCreationOptions.PreferFairness, _Scheduler)
                        UpdateUI.Wait()

                    Catch exp As Exception
                       'Do something here
                    End Try
                End If
            End Sub

        'Add an event handler for the Exited event
        AddHandler ProcessPool(_CurrentProcess).Exited,
                Sub(source As Object, ev As EventArgs)
                    _processexitcode = ProcessPool(_CurrentProcess).ExitCode
                    Console.WriteLine("The process has exited. Code: {0}  Time: {1}",
                    _processexitcode,
                    ProcessPool(_CurrentProcess).ExitTime)
                End Sub

        ProcessPool(_CurrentProcess).WaitForExit()
        ProcessPool(_CurrentProcess).Close()
        Return _processexitcode
    End Function, TaskCreationOptions.LongRunning, CancellationToken.None)

    Return If(_processexitcode = 0, 1, 0)
End Function

Note that, in the `Test_ParallelTasks()` method I used `CancellationToken.None` as a parameter. You can create a CancellationToken and pass it to the method if you need to Cancel the threads in a safe way. — Jimi, Jan 04 '18 at 14:43

Using Await in BackgroundWorker causes WorkerComplete event to be raised

1 Answers1

Linked