File parsing in perl reading first and last lines

Question

I have a file with first line

=== Verbose logging started: 1/3/2017  17:41:55  Build type: SHIP UNICODE 5.00.7601.00  Calling process: C:\Windows\SysWOW64\msiexec.exe ===

and last line

=== Verbose logging stopped: 1/3/2017  17:49:17 ===

I am interested in time fields in those lines (17:41:55 and 17:49:17), want to find the difference in time from start to stop.

I tried reading the file in an array and fetch first and last lines

my $last = pop (@array);
my $first = shift (@array);

But getting to time field in array is becoming difficult.

Could you please suggest any alternative way?

Answer 1

If you want to read the first and last line of a potentially very large log file, you shouldn't slurp it all into an array as it may consume a lot of memory. Instead, just read the first and last lines.

You can read the first line easy enough.

 use v5.10;
 use strict;
 use warnings;
 use autodie;

 open my $fh, $logfile;
 my $first = <$fh>;

You can read the last line by using seek to jump to the end of the file and then reading backwards in chunks with read until you get a whole line. That can get complicated. Fortunately there's File::ReadBackwards to do that for you.

use Carp;
use File::ReadBackwards;

my $backwards = File::ReadBackwards->new( $logfile )
    or croak "Can't open $logfile: $!";
my $last = $backwards->readline;

Note that if there's any stray newlines at the end of the file those will be the last line, so you might want to continue reading until you get what you're looking for.

# Read lines backwards until we get something that
# contains non-whitespace.
while( my $last = $backwards->readline ) {
    last if $last =~ /\S+/;
}

---

Here's a simpler, but slower (for large files) way to get the first and last lines. Read the first line as before, then read each line but only keep the last one.

my $last;
while( my $line = <$fh> ) { $last = $line }

It still has to read the whole file, but it only keeps the last one in memory.

---

Once you have that, you can parse the line and turn it into a Time::Piece object to work with it easier.

# === Verbose logging started: 1/3/2017  17:41:55 ... ===
# === Verbose logging stopped: 1/3/2017  17:49:17 ===
sub log_time {
    my $line = shift;

    # This captures the 1/3/2017  17:49:17 part
    my($datetime) = $line =~
        /^=== Verbose logging (?:started|stopped):\s*(\d+/\d+/\d+\s+\d+:\d+:\d+)/;

    # Parse it into a Time::Piece object.
    return Time::Piece->strptime($datetime, "%m/%d/%Y %H:%M:%S");
}

strptime is a function used by many languages to parse dates (string parse time). strftime (string format time) is used to format dates. They share the same mini language. Have a look at the strftime docs to understand what's going on there.

Once you have that, you can get the difference in seconds by subtracting them.

my $start = log_time($first);
my $end   = log_time($last);

say "Seconds elapsed: ".$end - $start;

Answer 2

I have a slightly less sophisticated approach to Schwern which is to use the Unix commands:

#!/usr/bin/perl

use strict;
use English;

my $first=`head -1 $ARGV[0]`;
my $last=`tail -1 $ARGV[0]`;

print "$first\n";
print "$last\n";