how to get X lines back with a streamreader

Question

Trying to implement a custom tail-like function, after checking several examples i've come to the bellow code, which works quite well (doesn't load whole file to read the X ending lines, works for network paths ...)

the problem I have is I don't how to move the stream pointer says 10 lines before its current position ?

as a workaround i move the pointer 1024 bytes before current position , but i don't know how much lines this is really concerning.

$sr=New-Object System.IO.StreamReader($fs)
$lastPosition=$sr.BaseStream.Length # final position of the file
$currentPosition=$lastPosition - 1024

Can anyone point me to right direction please ?

Here is the complete code :

function tail{
    [cmdletBinding()]
    Param(
        [Parameter(Position=0, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [System.String]
        $filename, # path
        [int]$n=10, # number of lines to output
        [switch]$continous, # continue to monitor for changes ?
        [switch]$hilight  # hilight lines containing keywords ?
    )

    # Initialising stuff
    $hilightError=@("erreur","error","fatal","critic")
    $hilightWarning=@("attention","warning","notice")
    $errorColor="red"
    $warningColor="darkyellow"

    if ( (test-Path $filename) -eq $false){
        write-Error "Cant read this file !"
        exit
    }

    function tailFile($ptr){
        # return each line from the pointer position to the end of file
        $sr.BaseStream.Seek($ptr,"Begin")
        $line = $sr.ReadLine() 
        while ( $line -ne $null){
            $e=$w=$false

            if( $hilight){
                $hilightError | %{ $e = $e -or ($line -match $_) } # find error keywords ?
                if( $e) {wh $line -ForegroundColor $errorColor }
                else{ 
                    $hilightWarning | %{ $w = $w -or ($line -match $_ ) } # find warning keywords ?
                    if($w){ wh $line -ForegroundColor $warningColor }
                    else{ wh $line}
                }
            }
            else{ #no hilight
                wh $line
            }
            $line = $sr.ReadLine() 
        }
    }

    # Main 
    $fs=New-Object System.IO.FileStream ($filename,"OpenOrCreate", "Read", "ReadWrite",8,"None") # use ReadWrite sharing permission to not lock the file
    $sr=New-Object System.IO.StreamReader($fs)

    $lastPosition=$sr.BaseStream.Length # final position of the file
    $currentPosition=$lastPosition - 1024 # take some more bytes  (to get the last lines)

    tailfile $currentPosition

    if($continous){
        while(1){
            start-Sleep -s 1
            # have the file changed ?
            if ($sr.BaseStream.Length -eq $lastPosition){
                write-verbose "no change..."
                continue
            }
            else {
                tailfile $lastPosition
                $lastPosition = $sr.BaseStream.Position
            write-Verbose "new position $lastPosition"
            }
        }
    }
    $sr.close()
}

I think you can take hints from this answer for c#: http://stackoverflow.com/a/4619770/520612 — CB., Dec 14 '12 at 12:15
Do you know that in V3 `get-content` has a `-tail` parameter? — CB., Dec 14 '12 at 13:32
@Christian yes thanks and with `-wait ` that's what i'm trying to reproduce, but i'd like to add custom functionality such as hilighting — Loïc MICHEL, Dec 14 '12 at 13:36
Are you already gone here (Keith Hill's Blog): http://rkeithhill.wordpress.com/2009/01/28/tail-content-%E2%80%93-better-performance-for-grabbing-last-lines-from-large-ascii-log-files/ — CB., Dec 14 '12 at 13:48
yep, got there I wish I knew how Keith did for the get-filetail cmdlet of pscx but this cmdlet is compiled into a dll ... — Loïc MICHEL, Dec 14 '12 at 13:53
Yes nut you can use dotPeek to decompile it ;) Isn't it obfuscate.. — CB., Dec 14 '12 at 14:56
@Christian didnt knew this tool. Thank you very much for all your efforts — Loïc MICHEL, Dec 14 '12 at 14:58
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/21147/discussion-between-kayasax-and-christian) — Loïc MICHEL, Dec 14 '12 at 17:19

score 1 · Answer 1 · answered Dec 14 '12 at 17:18

1

You can look at the PSCX Get-FileTail implementation here but it doesn't handle Unicode at all. And I'm sure there are other ways it could stand to be improved. :-) Suggestions are welcome.

answered Dec 14 '12 at 17:18

Keith Hill

194,368
42
353
369

so what thinks the expert ? Does it handle unicode correctly ? – Loïc MICHEL Dec 17 '12 at 18:11

score 1 · Accepted Answer · edited Nov 12 '14 at 15:45

thanks to Christian and Keith for your tips. Finally I've decided just to get backward in the streamreader until it returns enough readline(). I let the possibility to specify the codepage and seems to be ok with unicode. Here is the code if anyone is interested in

function tail{
    [cmdletBinding()]
    Param(
        [Parameter(Position=0, Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [System.String]
        $filename, # path
        [int]$n=10, # number of lines to output,
        [Alias("cp")]
        $codepage=65001,#utf8
        [Alias("f")]
        [switch]$continous, # continue to monitor for changes ?
        [switch]$hilight  # hilight lines containing keywords ?
    )
   # Initialising stuff
    $hilightError=@("erreur","error","fatal","critic")
    $hilightWarning=@("attention","warning","notice")
    $errorColor="red"
    $warningColor="yellow"
    [System.Text.Encoding]$enc = [System.Text.Encoding]::GetEncoding($codepage)

    function render($render){
        $e=$w=$false
        if( $hilight){
            foreach ($line in $render){
                $hilightError | %{ $e = $e -or ($line -match $_) } # find error keywords ?
                if( $e) {wh $line -ForegroundColor $errorColor }
                else{ 
                    $hilightWarning | %{ $w = $w -or ($line -match $_ ) } # find warning keywords ?
                    if($w){ wh $line -ForegroundColor $warningColor }
                    else{ wh $line}
                }
                $e=$w=$false
            }
        }
        else{ #no hilight
            wh $render
        }
    }


    function TailFileBeforeEnd{
    #try to find $n lines before eof

        $buffer=1024
        $ptr=$lastPosition #start at the end of the file  
        $found=0

        while($ptr -gt 0 -and $found -lt $n){
            $ptr-=$buffer 
            if ($ptr -le 0){$ptr=0}
            $sr.BaseStream.Seek($ptr,"Begin")|out-null #step backward
            $line = $sr.ReadLine()
            $found=0
            $output=@()
            $render=@()

            while ( $line -ne $null){ #read to the end
                $output+=$line
                $found++
                $line=$sr.ReadLine() 
            }
            if($found -ge $n){ #found enough lines
                Write-Verbose "OK found $found / $n"
                foreach($i in ($output.length - $n)..($output.length)){ #take only lines needed
                    $render+=$output[$i]
                }
                continue

            }
            else{ #move backward and retry to find lines
                Write-Verbose "not enough line ($found displayed)"
                $ptr-=$buffer
                if ($ptr -le 0) { #eof without finding suffisant lines
                    $ptr=0
                    Write-host "not enough line ($found displayed)"
                    $render=$output
                }
            }
         }
    render $render
    }


    function tailFile($ptr){
        # return each line from the pointer position to the end of file
        $render=@()
        $sr.BaseStream.Seek($ptr,"Begin")|out-null
        $line = $sr.ReadLine() 
        while ( $line -ne $null){
           $render+=$line
           $line = $sr.ReadLine() 
        }
        render $render
    }

    # Main loop


    # use ReadWrite sharing permission to not lock the file
    $fs=New-Object System.IO.FileStream ($filename,"OpenOrCreate", "Read", "ReadWrite",8,"None") 
    $sr=New-Object System.IO.StreamReader($fs, $enc)
    $lastPosition=$sr.BaseStream.Length 

    tailFileBeforeEnd 

    if($continous){
        while(1){
            start-Sleep -s 2
            # has the file changed ?
            if ($sr.BaseStream.Length -eq $lastPosition){
                write-verbose "no change..."
                continue
            }
            else {
                tailfile $lastPosition
                $lastPosition = $sr.BaseStream.Position
                write-Verbose "new position $lastPosition"
            }
        }
    }
    $sr.close()
}

how to get X lines back with a streamreader

2 Answers2