Parse IIS log file - is there an alternative to LogParser

Question

I need to parse an IIS log file. Is there any alternative to LogParser, a simple class to query a log file ?

I only need to know how many request I receive between 2 dates.

Here is an example of iis log file :

#Software: Microsoft Internet Information Services 7.5
#Version: 1.0
#Date: 2014-08-26 12:20:57
#Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
2014-08-26 12:20:57 W3SVC1 QXXXSXXXX 172.25.161.53 POST /XXXX/XXX/XXXX/XXXXX/1.0/XXXX/XXXXXXXX/xxxxxx.svc - 443 - 999.99.999.999 HTTP/1.1 - - - xxxx.xxxx.xxx.xxx.xxxx.xxxx.xxx.com 200 0 0 4302 5562 1560

Can you show us the structure and sample rows of the IIS logs? PowerShell and regex looks like a good ad hoc solution here. — Thomas Stringer, Aug 20 '15 at 14:11

Alex Nolasco · Answer 1 · 2017-07-25T15:28:31.477

It's 2017 and the LogParser is still closed source. Moreover, all the instrumentation provided by cloud solutions appears to be making the need for parsing IIS logs a thing of the past. But since I am also dealing with legacy apps, I wrote this simple parser using .NET core.

using System;
using System.IO;
using W3CParser.Extensions;
using W3CParser.Instrumentation;
using W3CParser.Parser;

namespace W3CParser
{
    class Program
    {
        static void Main(string[] args)
        {            
            var reader = new W3CReader(File.OpenText(args.Length > 0 ? args[0] : "Data/foobar.log"));

            using (new ConsoleAutoStopWatch())
            {
                foreach (var @event in reader.Read())
                {
                    Console.WriteLine("{0} ({1}):{2}/{3} {4} (bytes sent)",
                                      @event.Status.ToString().Red().Bold(),
                                      @event.ToLocalTime(),
                                      @event.UriStem.Green(),
                                      @event.UriQuery,
                                      @event.BytesSent);
                }
            }
        }
    }
}

Source code: https://github.com/alexnolasco/32120528

You should make a nuget from it IMHO – Simon Mourier Aug 11 '17 at 10:18 — Simon Mourier, Aug 11 '17 at 10:18

Anas · Accepted Answer · 2015-08-21T01:06:49.463

11

You can use Tx (LINQ to Logs and Traces) , you can install it via nuget

and use it like this:

var iisLog = W3CEnumerable.FromFile(pathToLog);
int nbOfLogsForLastHour = iisLog.Where(x => x.dateTime > DateTime.Now.AddHours(-1)).Count();

If the log file is used by another process, you can use W3CEnumerable.FromStream

edited Aug 21 '15 at 01:06

answered Aug 21 '15 at 01:01

Anas

5,622
5
39
71

2

Doesn't work, doesn't compile, does compile as framework 4.5 when set to framwork 4.0 (resets itselfs if changed), and doesn't seem to be a simple log parser. Outputs binaries to c:\bin\debug... BS. Why does a LogParser need unsafe code ? Why does a logparser need platform dependencies ? Why does it generate binaries for 4.5 when set to 4.0 ? Why does 1 logparser need to be 6 dlls ? Why does it even need Linq ? BS. Big major-leage BS. The only thing that could possibly be a worse example of BS code would be using Microsoft Log-Parser with COM-Interop. – Stefan Steiger Nov 02 '16 at 08:04

score 5 · Answer 3 · edited Jun 25 '20 at 21:47

5

You can use IISLogParser , and install it via nuget, it has support for large files (> 1Gb)

List<IISLogEvent> logs = new List<IISLogEvent>();
using (ParserEngine parser = new ParserEngine([filepath]))
{
    while (parser.MissingRecords)
    {
        logs = parser.ParseLog().ToList();
    }
}

edited Jun 25 '20 at 21:47

soccer7

3,547
3
29
50

answered Nov 23 '18 at 14:53

Kabindas

802
9
6

lii5a · Answer 4 · 2019-11-11T09:15:57.967

If you're dealing with large volumes and/or dispersed locations of IIS log files, then SpectX is a handy tool for this because you don't have to ingest the logs and can run queries directly on multiple raw files. Avg processing speed per core - 350MB/sec.

It's not open source but the full-functionality 30-day trial is free.

Tutorials: Parsing IIS logs. Analyzing IIS logs - 20 sample queries.

To filter a time period, sort the logs by date and filter the period you need, e.g:

    | sort(date_time desc)
    | filter(date_time > T('2019-11-01 08:48:20.000 +0200')) 
    | filter(date_time < T('2019-11-05 11:48:20.000 +0200'));

score 0 · Answer 5 · answered Dec 15 '22 at 05:26

0

I use filter feature of CMTrace.exe tool (Refer screenshot):

answered Dec 15 '22 at 05:26

RBT

24,161
21
159
240

Parse IIS log file - is there an alternative to LogParser

5 Answers5