0

Currently I'm using this command on Linux:

grep Ban /var/log/fail2ban.log | grep -v 'Restore Ban' | sed 's/\s\s*/ /g' | cut -d" " -f8 | sort | uniq -c | sort -t ' ' -n -b

The log file looks like this:

2019-03-04 07:14:45,778 fail2ban.filter         [19052]: INFO    [sshd] Found 2*8.1*7.1*9.2*9
2019-03-04 07:14:46,412 fail2ban.actions        [19052]: NOTICE  [sshd] Ban 2*8.1*7.1*9.2*9
2019-03-04 07:15:04,708 fail2ban.actions        [19052]: NOTICE  [sshd] Unban 1*9.2*.2*4.1*6
...

The output looks like this:

8  1*2.2*6.1*1.1*5
12  3*.1*.*4.*6
18 1*5.2*8.2*5.4
19 1*2.2*6.1*1.1*4
72 3*.1*6.2*.9*

I already tried it with Get-Content but I don't understand all of the PowerShell syntax.

Ansgar Wiechers
  • 193,178
  • 25
  • 254
  • 328
Monty
  • 15
  • 2
  • 2
    If you don't understand some cmdlets yet you can always do a [Get-Help Get-Content -Full](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.management/get-content?view=powershell-5.1). You might read the help for [Select-String](https://learn.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/select-string?view=powershell-5.1). – Olaf Mar 06 '19 at 15:07
  • 4
    Please go find a PowerShell tutorial. SO is neither a free translation service, nor a replacement for familiarizing yourself with the basics of the language you're using. – Ansgar Wiechers Mar 06 '19 at 15:48

2 Answers2

2

Your Linux command packs a lot of functionality into a single pipeline.
Your lack of effort to solve the problem yourself notwithstanding, constructing an equivalent PowerShell command is an interesting exercise in contrasting a Unix-utilities solution with a PowerShell solution:

To set the scene, let me explain what your command does:

  • grep Ban /var/log/fail2ban.log case-sensitively finds lines that contain the word Ban in file /var/log/fail2ban.log and passes only those on.

  • grep -v 'Restore Ban' further (case-sensitively) filters out (-v) lines that contain the phrase 'Restore Ban'.

  • sed 's/\s\s*/ /g' replaces all (g) runs of 1 or more whitespace chars. (\s; in a modern regex dialect you'd use \s+) with a single space ...

  • ... which then allows cut -d" " -f8 to reliably extract the 8th field from each line from the resulting space-separated list (e.g., 2*8.1*7.1*9.2*9).

  • sort then lexically sorts the resulting lines, and uniq -c weeds out duplicates, while prepending each unique line with the count of duplicates (-c), with 1 indicating a unique line.

  • Finally, sort -t ' ' -n -b sorts the resulting lines numerically by duplicate count.

In short: your command filters a log file via regex matching, extracts the 8th field from each line, eliminates duplicates, and prints unique fields prefixed with their duplicate count, sorted by duplicate count in ascending order.


Below is a near-equivalent PowerShell command, which:

  • is more readable (and therefore, of necessity, more verbose)

  • involves fewer steps

  • ultimately offers much more flexibility, due to:

    • sending objects through the pipeline, not just text that must often be (re)parsed - it is this feature that constitutes PowerShell's evolutionary quantum leap from traditional shells.
    • far superior language features (compared to POSIX-like shells such as bash) that can easily be woven into a pipeline.

That said, the price you pay for the increased power is performance:

  • Directly comparable commands perform much better using Unix utilities, though the usually higher level of abstraction and flexibility provided by PowerShell cmdlets may make up for that.

Here's the command, with the roughly corresponding Unix-utility calls in comments:

Select-String -CaseSensitive '(?<!Restore )Ban' /var/log/fail2ban.log | #grep,grep -v
  ForEach-Object { (-split $_.Line)[7] } | # sed, cut -f8
    Group-Object | # uniq -c 
      Select-Object Count, Name | # construction of output *objects*
        Sort-Object Count, Name # sort, sort -n

The command outputs objects with a .Count (duplicate count) and .Name property (the 8th field from the log file), which:

  • allow for robust additional processing (no parsing of textual output needed).
  • render in a friendly manner to the console (see below).

Example output:

Count Name
----- ----
    8 1*2.2*6.1*1.1*5
   12 3*.1*.*4.*6
   18 1*5.2*8.2*5.4
   19 1*2.2*6.1*1.1*4
   72 3*.1*6.2*.9*

For an explanation of the command, consult the following help topics, which are also available locally, as part of a PowerShell installation, via the Get-Help cmdlet:

mklement0
  • 382,024
  • 64
  • 607
  • 775
0
((Get-Content "fail2ban.log") -cmatch "(?<!Restore )Ban" | Select-String -Pattern "[0-9.*]+$" -AllMatches).matches.value | Group-Object | foreach {"$($_.count) $($_.name)"}

Get-Content here is grabbing each line of the fail2ban.log file. The -cmatch operator is performing a case-sensitive regex match. The regex pattern looks for the string Ban with a negative look behind of string Restore . The Select-String is looking for a regex pattern at the end of each line that has characters in the set (0123456789.*). The matches.value property outputs only the matched strings from the regex. Group-Object groups each identically matched value as property Name and adds a count property. Since the OP was capturing count, I decided to use Group-Object to easily get that. The foreach is simply doing formatting to match the output presentation of the OP.

AdminOfThings
  • 23,946
  • 4
  • 17
  • 27
  • I added some information. My intention was to give a one liner that would provide identical output based on his data set. I did not attempt to provide the identical PowerShell equivalent to each Linux command provided by the poster. So if that’s what the OP wanted, then my answer isn’t the best. My answer is an alternative approach if the data is consistent and you like regex. – AdminOfThings Mar 07 '19 at 00:38
  • im still thankfull for your answer. i would have researched the commands by myself. yet i dont want to learn powershell, i just needed this one command. so thanks for your help :) – Monty Mar 08 '19 at 07:24