Real Time monitoring for errors in log files in windows

Question

I have multiple windows services deployed on my servers. I want to implement a PowerShell script which can do real time monitoring of logs of these services. It must look for keywords(Eg. Error, Exception) in the log file and as soon as there is any error, the script should send a notification to a preconfigured email address. I did basic search on the web and could find some freeware applications which can do this but I am not keen on installing those on the servers. If this can be done by a basic PowerShell script or a batch script and can run in background, it'll be great.

I've found the Get-Content and Type -wait commands which can watch the file in real time

Get-Content error.log -wait | where { $_ -match "ERROR" }

I'll really appreciate any assistance on the email notification part and if you could add some web links which may help.

A bit of complexity is that the log file will not be constant and a new log file is created every day and the script should automatically identify the latest file based on the file name or creation date etc.

File Name format is 8_05_2021.txt, 9_05_2021.txt, 10_05_2021.txt

Answer 1

If my logic is right I think this should work, this script would run indefinitely.

For sending Mails in PowerShell you have two options that I'm aware of, one is using the cmdlet designed for that: Send-MailMessage

However, this is important to be aware of:

Warning

The Send-MailMessage cmdlet is obsolete. This cmdlet does not guarantee secure connections to SMTP servers. While there is no immediate replacement available in PowerShell, we recommend you do not use Send-MailMessage. For more information, see Platform Compatibility note DE0005.

You can find the second option here using Net.Mail.MailMessage.

Now for the code of the script, here is something you can use:

# Define the full path of your logs folder
$logsFolder = 'fullPath\to\logsFolder'

# Function for monitoring and retrieving the newest log file Full Path
function Get-NewestLog ($LogPath) {
    Get-ChildItem $LogPath -Filter *.txt | Sort-Object CreationTime -Descending |
        Select-Object -First 1 -ExpandProperty FullName
}


# Get the newest log file
$logFilePath = Get-NewestLog -LogPath $logsFolder

while($true) {
    # If we don't have today's date stored
    # or the update trigger is True
    if($updateDate -or -not $today) {
        $today      = [datetime]::Today
        $updateDate = $false
    }
    
    if($today -lt [datetime]::Today) {
        # Trigger our previous condition
        $updateDate = $true

        # Get the new log file for this day
        $logFilePath = Get-NewestLog -LogPath $logsFolder
    }

    if((Get-Content $logFilePath -Raw) -match 'Error') {
        # Send mail message goes here
    }

    Start-Sleep -Seconds 60
}

It is important to note that, this would spam your inbox every minute if there is an error in the log file so it will probably be a good idea to add a new condition in this block:

if((Get-Content $logFilePath -Raw) -match 'Error') { 
    ....
}

For example something like this:

if((Get-Content $logFilePath -Raw) -match 'Error' -and -not $emailSentThisDay) {
    # Send mail message goes here

    # Here you set this bool to True so you don't get spammed :D
    $emailSentThisDay = $true
}

If this is something you will consider then you will need to reset the $emailSentThisDay bool every new day so:

if($today -lt [datetime]::Today) {
    # Trigger our previous condition
    $updateDate = $true

    # Reset the antispam bool if this is a new day
    $emailSentThisDay = $false
    
    # Get the new log file for this day
    $logFilePath = Get-NewestLog -LogPath $logsFolder
}

Answer 2

Note: The solution below only monitors log-file additions going forward in time, it doesn't consider old log files and it doesn't keep persistent track of what log file up to what line has already been processed - such logic would require (substantial) additional effort.

You can use the following approach:

• Use background jobs (created with Start-Job) or, preferably, thread [background] jobs (created with Start-ThreadJob) to monitor additions to a given log file.

• In the foreground, run a loop that periodically:

  • polls the job for new log-file content of interest and sends an email, if found.
  • checks for the presence of a new log file, and, if found, terminates the current job and starts a new one for the new log file.

Note:

• You can use Send-MailMessage to send emails, but note that this cmdlet is considered obsolete, because it "does not guarantee secure connections to SMTP servers." That said, if that isn't a concern in your case, it works, and, given PowerShell's commitment to backward compatibility, the cmdlet is unlikely to ever be removed.

• The code runs indefinitely; when run interactively, you can terminate it with Ctrl-C, but in an automated invocation scenario such as a scheduled task you'll have to terminate the task.

$logFileDir = '.'
$job = $null
$currLogFile = $null

while ($true)
{

  # Check for new content of interest from the background job.
  if ($job -and ($result = Receive-Job $job)) {
    # Send an email here, e.g.:
    # Send-MailMessage -SmtpServer exchange.example.com -From alerts@example.com -To jdoe@example.com -Subject 'Error' -Body $result
  }

  # See if a new log file has appeared.
  # For simplicity, go by creation date.
  # If a new file only appears once a day, you could keep track
  # of the current calendar date and only look when it changes.
  $newLogFile = (Get-ChildItem -File "$logFile/*_*_*.txt" | 
                  Sort-Object -Descending CreationTime | 
                    Select-Object -First 1).FullName
  
  if ($newLogFile -ne $currLogFile) {

    # If there's a current job for the previous log file, terminate it now.
    if ($job) { Remove-Job $job -Force }

    # Create a job for the new log file.
    $currLogFile = $newLogFile
    $job = Start-Job {
       # Wait indefinitely for content to be added to the file,
       # and output lines matching the string of interest.
       Get-Content -LiteralPath $using:currLogFile -Wait |
         Where-Object { $_ -match "ERROR" }
    }

  }

  # Sleep a little.
  Start-Sleep -Seconds 1
}