Keep track of the form activation state by listening to the Activate and Deactivate events. One complication is that the form will be de-activated when you click the icon. Solve that by recording the time it happened. Like this:
Public Class Form1
Private IsActivated As Boolean
Private DeActivation As DateTime
Protected Overrides Sub OnActivated(e As System.EventArgs)
IsActivated = True
MyBase.OnActivated(e)
End Sub
Protected Overrides Sub OnDeactivate(e As System.EventArgs)
IsActivated = False
DeActivation = DateTime.Now
MyBase.OnDeactivate(e)
End Sub
Private Sub NotifyIcon1_MouseClick(sender As System.Object, e As System.Windows.Forms.MouseEventArgs) Handles NotifyIcon1.MouseClick
If Me.WindowState = FormWindowState.Minimized Then
Me.WindowState = FormWindowState.Normal
Else
If IsActivated OrElse DateTime.Now - DeActivation < New TimeSpan(0, 0, 1) Then
Me.WindowState = FormWindowState.Minimized
Else
Me.Activate()
End If
End If
End Sub
End Class
This is otherwise a fairly unpleasant hack around having to set the form's ShowInTaskbar property to False. It is possible to have a taskbar button and still keep the form invisible at startup. Check this answer for the approach.