FileStream from the System.IO namespace instead of Get-Content -Tail

Question

I want to use a FileStream from the System.IO namespace instead of Get-Content cmd-let. How can I do that ?

thanks,

$fromaddress = "filemon@contoso.com"
$emailto = "IT@contoso.com"
$SMTPSERVER = "xx.xx.xx.xx"
$global:FileChanged = $false 
$folder = "C:\temp" 
$filter = "log.log" 
$watcher = New-Object IO.FileSystemWatcher $folder,$filter -Property @{ IncludeSubdirectories = $false EnableRaisingEvents = $true }
Register-ObjectEvent $Watcher "Changed" -Action {$global:FileChanged = $true} > $null

while ($true)
{ 
    while ($global:FileChanged -eq $false){ 
        Start-Sleep -Milliseconds 100 
    }

    if($(Get-Content -Tail 1 -Path $folder\$filter).Contains("Finished."))
    {
        Send-mailmessage -from $fromaddress -to $emailto -subject "Log changed" -smtpServer $SMTPSERVER -Encoding UTF8 -Priority High
    }

    # reset and go again
    $global:FileChanged = $false
}

EDIT 1:

$fromaddress = "filemon@contoso.com"
$emailto = "IT@contoso.com"
$SMTPSERVER = "xx.xx.xx.xx"
$global:FileChanged = $false 
$folder = "C:\tmp" 
$filter = "log.log" 
$watcher = New-Object IO.FileSystemWatcher $folder,$filter -Property @{ IncludeSubdirectories = $false ; EnableRaisingEvents = $true }
Register-ObjectEvent $Watcher "Changed" -Action {$global:FileChanged = $true} > $null

function Read-LastLine ([string]$Path) {
    # construct a StreamReader object
    $reader   = [System.IO.StreamReader]::new($path)
    $lastLine = ''
    while($null -ne ($line = $reader.ReadLine())) {
        $lastLine = $line
    }

    # clean-up the StreamReader
    $reader.Dispose()

    # return the last line of the file
    $lastLine
}

while ($true)
{ 
    while ($global:FileChanged -eq $false){ 
        Start-Sleep -Milliseconds 100 
    }
    
    $logFile = $Event.SourceEventArgs.FullPath

    if ((Read-LastLine -Path $logFile) -match "Finished.")
    {
       write-host "mail sending"
        Send-mailmessage -from $fromaddress -to $emailto -subject "Log changed" -smtpServer $SMTPSERVER -Encoding UTF8 -Priority High
    }

    # reset and go again
    $global:FileChanged = $false
}

Message:

MethodInvocationException: C:\monfile.ps1:15
Line |
  15 |      $reader   = [System.IO.StreamReader]::new($path)
     |      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | Exception calling ".ctor" with "1" argument(s): "The value cannot be an empty string. (Parameter 'path')"
InvalidOperation: C:\monfile.ps1:17
Line |
  17 |      while($null -ne ($line = $reader.ReadLine())) {
     |                       ~~~~~~~~~~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.
InvalidOperation: C:\monfile.ps1:22
Line |
  22 |      $reader.Dispose()
     |      ~~~~~~~~~~~~~~~~~
     | You cannot call a method on a null-valued expression.

Answer 1

Try following :

$SourceStream = [System.IO.FileStream]::new('c:\temp\test.txt',[System.IO.FileMode]::Open);
for($pos = $SourceStream.Length - 1; $pos -ge 0; $pos--)
{
   $SourceStream.Position = $pos
   $b = $SourceStream.ReadByte()
   if(($b -eq 0x0A) -or ($b -eq 0x0D)) { break;}
}
$length = $SourceStream.Length - $pos
$buffer = New-Object byte[]($length)
$SourceStream.Read([byte[]]$buffer, 0, $length)
$tail = [System.Text.Encoding]::UTF8.GetString($buffer, 0, $length)
Write-Host $tail

Answer 2

Not sure if this would be faster than using Get-Content -Tail 1, but you could create a helper function that uses a StreamReader to return the last line in the file.

Edit

Your code does not really respond to the event when it happens, but seems to rely on a global variable called FileChanged which you test inside a while loop.

The magic with a FileSystemWatcher is that you can just sit back and relax until the chosen event fires, in your case if something has changed in the log.log file. Only when such an event occurs, the scripblock defined in the Action parameter is executed, so here's a rewrite of your code using a helper function to read the last line of the log file.

(since my earliear answer, I have changed that function to only retrieve the last line that is not empty or whitespace only)

function Read-LastLine {
    [CmdletBinding()]
    param (
        [ValidateNotNullOrEmpty()]
        [Alias('FullName')]
        [string]$Path
    )
    # construct a StreamReader object
    $reader   = [System.IO.StreamReader]::new($path)
    $lastLine = [string]::Empty
    while($null -ne ($line = $reader.ReadLine())) {
        # do not store this line if it is empty or whitespace-only
        if (![string]::IsNullOrWhiteSpace($line)) {
            $lastLine = $line
        }
    }

    # clean-up the StreamReader
    $reader.Dispose()

    # return the last line of the file
    $lastLine
}

# create a splatting Hashtable for the Send-MailMessage cmdlet
$mailParams = @{
    From       = "filemon@contoso.com"
    To         = "IT@contoso.com"
    SmtpServer = "xx.xx.xx.xx"
    Subject    = "Log changed"
    Encoding   = 'UTF8'
    Priority   = 'High'
}

# define parameters for the FileSystemWatcher
$folder = 'C:\tmp'
$filter = 'log.log'

# to prevent the FileSystemWatcher to fire events twice, keep track of the last time
$LastEventTime = (Get-Date)

# define the Action scriptblock for the FileSystemWatcher
# this code which will be executed every time a file change is detected
$action = {
    # try to detect if this event fired twice
    if (($Event.TimeGenerated.Ticks - $script:LastEventTime.Ticks) -gt 100000) {
        $logFile = $Event.SourceEventArgs.FullPath
        if ((Read-LastLine -Path $logFile) -match "Finished") {
            $script:LastEventTime = $Event.TimeGenerated
            Write-Host "mail sending"
            Send-mailmessage @mailParams
        }
    }
}

# create the FileSystemWatcher and register the event
$watcher = New-Object IO.FileSystemWatcher $folder, $filter -Property @{
    IncludeSubdirectories = $false
    EnableRaisingEvents   = $true
    NotifyFilter          = [IO.NotifyFilters]::LastWrite
}
Register-ObjectEvent $Watcher -EventName 'Changed' -SourceIdentifier 'LogChanged' -Action $action

When you no longer need to watch the file, unregister the event and dispose of the watcher

Unregister-Event -SourceIdentifier 'LogChanged'
$watcher.Dispose()

Answer 3

There is one real problem with reading the last line of a log file that is being written to by software that you didn't create - How do you know it is only adding one line, or log entry, at a time? What you really need is something that keeps track of the log file and reads only new content as it is added. My solution to this is to create a LogFileReader class.

In a test script I created, I added to the top of the script the following lines taken, as is, from the question:

$fromaddress = "filemon@contoso.com"
$emailto = "IT@contoso.com"
$SMTPSERVER = "xx.xx.xx.xx"
$global:FileChanged = $false 
$folder = "C:\tmp" 
$filter = "log.log" 
$watcher = New-Object IO.FileSystemWatcher $folder,$filter -Property @{ IncludeSubdirectories = $false ; EnableRaisingEvents = $true }
Register-ObjectEvent $Watcher "Changed" -Action {$global:FileChanged = $true} > $null

Then added this experimental LogFileReader class. I created this class sometime in the last few weeks to address this issue. So far it works well and I'm not seeing it throw errors or have problems. But again, it is experimental and not thoroughly tested.

For this class:

1. Create an instance with the full path of the file, and optionally you can add the encoding. If no encoding is provided, then UTF8 is the default.
2. If you want to read the full file prior to reading new lines, then make a call to either ReadLines() or ReadText(), else, make a call to GoToEnd() to skip the current content of the file.
3. ReadLines() returns an array of strings. If you want to get each newly added line one at time, use this.
4. ReadText() returns a single string. If you want all the newly added content as a single string, use this.
5. When done, use Dispose() to release the StreamReader. Edit: Replaced the old Dispose() method with a new one that retrieves the FileStream from StreamReader's BaseStream property, disposes of the StreamReader, and then disposes of the FileStream.
6. If for some reason, after several reads, you want to read the file from the beginning, call GoToStart() and then the next read will get the entire file's content.
7. The class maintains a pointer that points to the end the last read, and with each call to ReadLines(), the pointer is moved forward the the end of the current read. ReadText()' calls ReadLines(), so the pointer is also advanced with each call to ReadText()`.

class LogFileReader: IDisposable {
    [string]$_logFile
    [int]$_position
    [IO.StreamReader]$_streamReader
    hidden [void]Initialize([string]$logFile, [Text.Encoding]$encoding) {
        $this._logFile = $logFile
        $this._position = 0
        $this._streamReader = if ([IO.File]::Exists($this._logFile)) {
            [IO.StreamReader]::new([IO.FileStream]::new( $logFile, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::ReadWrite), $encoding)
        } else {$null}
    }
    LogFileReader([string]$logFile) { $this.Initialize($logFile, [Text.Encoding]::UTF8) }
    LogFileReader([string]$logFile, [Text.Encoding]$encoding) { $this.Initialize($logFile, $encoding) }
    [void]GoToStart() { $this._position = 0 }
    [void]GoToEnd() { if($this._streamReader){$this._position = $this._streamReader.BaseStream.Length} }
    [string[]]ReadLines() {
        $Lines = @()
        if($this._streamReader) {
            $BaseLength = $this._streamReader.BaseStream.Length
            if ( $this._position -ne $BaseLength ) {
                if($this._position -gt $BaseLength ) { $this.GotoStart() }
                $null = $this._streamReader.BaseStream.Seek($this._position, [IO.SeekOrigin]::Begin)
                while ($null -ne ($Line = $this._streamReader.ReadLine())) {if($Line){$Lines += $Line}}
                $this._position = $this._streamReader.BaseStream.Position
            }
        }
        return $Lines
    }
    [string]ReadText() {
        $Return = ''
        $this.ReadLines() | ForEach-Object {$Return += "$_$([System.Environment]::NewLine)"}
        return $Return
    }
    [void]Dispose() {
        try{
            [IO.FileStream]$FileStream = $this._streamReader.BaseStream
            $this._streamReader.Dispose()
            $FileStream.Dispose()
        }
        finally{}
    }
}

The following test code was added to the end of the script. EDIT: It now has commented out code that allows catching Ctrl+C and Gracefully stopping in Powershell. Apparently, [console]::TreatControlCAsInput isn't available in all situation, even though the docs don't appear to give enough info to explain when it isn't. It also includes 3 version of the reading of the new content. A foreach {} version with if for catching "Finished." line, and 2 commented out versions to give example uses.

The Send-mailmessage line is commented out, and an extra Write-Host "[$Line]" added for testing purposes.

$LogFileReader = [LogFileReader]::new("$folder\$filter", [Text.Encoding]::UTF8)
$LogFileReader.GoToEnd()

#[console]::TreatControlCAsInput = $true
:TrappedForever while ($true) {
    while ($global:FileChanged -eq $false){
        Start-Sleep -Milliseconds 100
        #if ($Host.UI.RawUI.KeyAvailable -and (3 -eq [int]$Host.UI.RawUI.ReadKey("AllowCtrlC,IncludeKeyUp,NoEcho").Character)) {break TrappedForever}
    }
    foreach ($Line in $LogFileReader.ReadLines()) {
        Write-Host "[$Line]"
        if ($Line -match "Finished.") {
            write-host "mail sending"
            #Send-mailmessage -from $fromaddress -to $emailto -subject "Log changed" -smtpServer $SMTPSERVER -Encoding UTF8 -Priority High
        }
    }
    #$LogFileReader.ReadLines() | ForEach-Object {Write-Host "[$_]"}
    #Write-Host ($LogFileReader.ReadText()) -NoNewline

    # reset and go again
    $global:FileChanged = $false
}
$watcher.EnableRaisingEvents = $false
$watcher.Dispose()
$LogFileReader.Dispose()
Write-Host "All Done!"

Answer 4

No IO.FileSystemWatcher Version:

While looking up references for my previous answer, I discovered that the LogFileReader class I created was based on C# code, How to read a log file which is hourly updated in c#?, that is remarkably similar to the code presented in the original code.

This LogFileWatcher class is similar to the previous LogFileReader class, but stripped down to the basics, and with a Dirty() method that indicates when more lines are available:

class LogFileWatcher: IDisposable {
    [IO.StreamReader]$_streamReader
    [int]$_position
    hidden [void]Initialize([string]$logFile, [Text.Encoding]$encoding) {
        $this._position = 0
        $this._streamReader = if ([IO.File]::Exists($logFile)) {
            [IO.StreamReader]::new([IO.FileStream]::new( $logFile, [IO.FileMode]::Open, [IO.FileAccess]::Read, [IO.FileShare]::ReadWrite), $encoding)
        } else {$null}
    }
    LogFileWatcher([string]$logFile) { $this.Initialize($logFile, [Text.Encoding]::UTF8) }
    LogFileWatcher([string]$logFile, [Text.Encoding]$encoding) { $this.Initialize($logFile, $encoding) }
    [void]GoToEnd() { if($this._streamReader){$this._position = $this._streamReader.BaseStream.Length} }
    [bool]Dirty() {
        return $this._position -ne $this._streamReader.BaseStream.Length
    }
    [string[]]ReadLines() {
        $Lines = @()
        if($this._streamReader -and $this.Dirty()) {
            $null = $this._streamReader.BaseStream.Seek($this._position, [IO.SeekOrigin]::Begin)
            while ($null -ne ($Line = $this._streamReader.ReadLine())) {if($Line){$Lines += $Line}}
            $this._position = $this._streamReader.BaseStream.Position
        }
        return $Lines
    }
    [void]Dispose() {
        try{
            [IO.FileStream]$FileStream = $this._streamReader.BaseStream
            $this._streamReader.Dispose()
            $FileStream.Dispose()
        }
        finally{}
    }
}

Example use, a hybrid of the C# code, and code in previous answer:

$fromaddress = "filemon@contoso.com"
$emailto = "IT@contoso.com"
$SMTPSERVER = "xx.xx.xx.xx"
$folder = "C:\tmp"
$filter = "log.log"

$LogFileWatcher = [LogFileWatcher]::new("$folder\$filter", [Text.Encoding]::UTF8)
$LogFileWatcher.GoToEnd()

#[console]::TreatControlCAsInput = $true
while ($true) {
    Start-Sleep -Milliseconds 100
    #if ($Host.UI.RawUI.KeyAvailable -and (3 -eq [int]$Host.UI.RawUI.ReadKey("AllowCtrlC,IncludeKeyUp,NoEcho").Character)) {break}
    if($LogFileWatcher.Dirty) {
        foreach ($Line in $LogFileWatcher.ReadLines()) {
            if ($Line -match "Finished.") {
                write-host "mail sending"
                Send-mailmessage -from $fromaddress -to $emailto -subject "Log changed" -smtpServer $SMTPSERVER -Encoding UTF8 -Priority High
            }
        }
    }
}
#$LogFileWatcher.Dispose()
#Write-Host "That's all Folks!"

Also, if memory serves me correctly, it was What's the least invasive way to read a locked file in C# (perhaps in unsafe mode)? that gave me the clues needed to eventually find the above link with code I used.