Read two lines before last line of text file

Question

I have a log file I am trying to get some info out of. The information I need is on the line prior to last line and also the very last line is/could be blank. So it is actually line before last line or two before last line if last line is blank.

I know how to get to last line of the file using:

var lastLine = File.ReadLines("SomeFile.log").Last();

I can also use Linq to skip lines using .skipWhile() or .skip(1) but not going backward.

I am not sure how to get to the line I need. This is a sample of last few lines of the log file (with last line being blank):

2021/05/02 23:47:57:008989 send_status_message(2) Info: "Stream status heartbeat sent: [SY  1.3.2       ]"
2021/05/02 23:47:57:225172 send_status_message(2) Info: "Received heartbeat response: [S               ]"
2021/05/03 00:00:00:045055 set_log_dir(2) Info: "Changing log directory to /abc/def/logs/2021-05-03."
<blank-line>    

I am trying to get the timestamp on that line (i.e. 2021/05/02 23:47:57:225172).

Answer 1

File.ReadLines("SomeFile.log").Last(); will iterate over all lines and keep the last. That can be expensive for large files. At least it doesn't keep all of them in memory.

A faster alternative would be to read the last X bytes, convert them to string and split it into lines. Which isn't as easy as it sounds if you have UTF8 files, as the chunk may miss the first byte(s) of the first character. This question asks how to do this and UTF8 is left as an excercise to the reader.

To retrieve the last N items in an IEnumerable<T> you can use the TakeLast method introduced by .NET Core :

var lastLines = File.ReadLines("SomeFile.log").TakeLast(2);

There's also SkipLast, so if you wanted the second from last line, you could use:

var secondLast = File.ReadLines("SomeFile.log").TakeLast(2).SkipLast(1);

For just 1 line though, TakeLast(2).FirstOrDefault() would be enough.

For .NET Framework, you can use something like this answer's code or this one to iterate and keep the last N lines:

public static IEnumerable<T> TakeLast<T>(this IEnumerable<T> source, int count)
{
    if (source == null) { throw new ArgumentNullException("source"); }

    Queue<T> lastElements = new Queue<T>();
    foreach (T element in source)
    {
        lastElements.Enqueue(element);
        if (lastElements.Count > count)
        {
            lastElements.Dequeue();
        }
    }

    return lastElements;
}

This code needs a slight change to become a SkipLast(), by returning the dequeued items instead of discarding them:

public static IEnumerable<T> SkipLast<T>(this IEnumerable<T> source, int count)
{
    if (source == null) { throw new ArgumentNullException("source"); }

    Queue<T> lastElements = new Queue<T>();
    foreach (T element in source)
    {
        lastElements.Enqueue(element);
        if (lastElements.Count > count)
        {
            var head=lastElements.Dequeue();
            yield return head;
        }
    }
}

Answer 2

Using C#8's range operator

If you already have the array in memory and can use C# 8, you could do this:

var Lines = File.ReadAllLines("SomeFile.log");
var SecondToLast = Lines[^2];

Without C#8.

Alternatively, as mentioned by Tim, you could do the arithmetic on the indexer:

var Lines = File.ReadAllLines("SomeFile.log");
var SecondToLast = Lines[Lines.Length - 2];

Comments based edit. From your comments, it seems like you're not quite sure of how many empty lines you'll get. If that's the case, you might be better off using a more general approach, such as this:

    static string FirstNotEmpty(string[] Lines, bool BottomUp = false)
    {
        if (BottomUp)
        {
            for (int i = Lines.Length - 1; i >= 0; i--)
            {
                var CurrentLine = Lines[i];
                if (!string.IsNullOrWhiteSpace(CurrentLine))
                    return CurrentLine;
            }
        }
        else
        {
            for (int i = 0; i <= Lines.Length-1; i++)
            {
                var CurrentLine = Lines[i];
                if (!string.IsNullOrWhiteSpace(CurrentLine))
                    return CurrentLine;
            }
        }
        return null; //Or something else.
    }

In your case, you'd call it like this:

var FirstNotEmptyLine = FirstNotEmpty(Lines, BottomUp: true);

You could also preemptively remove empty lines from your array:

var WithoutEmptyLines = Lines.Where(x => !string.IsNullOrWhiteSpace(x));

And then "safely" get the last line.

Answer 3

Maybe you could use this extension method:

public static class EnumerableExtensions
{
    public static T GetLastItem<T>(this IEnumerable<T> seq, int countFromEnd)
    {
        if(seq is IList<T> list) return list[^countFromEnd];
        using var enumerator = seq.Reverse().GetEnumerator();
        while(enumerator.MoveNext())
        {
            if(--countFromEnd == 0) return enumerator.Current;
        }
        throw new ArgumentOutOfRangeException();
    }
}

Usage:

var secondLastLine = File.ReadLines("SomeFile.log").GetLastItem(2);

If you don't use C#8, so you can't use Ranges, replace return list[^countFromEnd] with return list[list.Count - countFromEnd].

Answer 4

Something like this might do it for you

 var lines = System.IO.File.ReadLines(@"SomeFile.log");
 var secondLastIdx = lines.Count() - 2;
 var secondlast = lines.Skip(secondLastIdx ).First();

You might need to use something better to figure out secondLastIdx