33

I'm trying to parse a logfile using grok

Each line of the logfile has fields separated by commas:

13,home,ABC,Get,,Private, Public,1.2.3 ecc...

I'm using match like this: match => [ "message", "%{NUMBER:requestId},%{WORD:ServerHost},%{WORD:Service},...

My question is: Can I allow optional field? At times some of the fileds might be empty ,,

Is there a pattern that matches a string like this 2.3.5 ? ( a kind of version number )

alpa
  • 345
  • 1
  • 4
  • 6
  • Apart from the grok filter the [csv filter](http://logstash.net/docs/latest/filters/csv) is made for parsing this type of data. – Magnus Bäck May 09 '15 at 18:22

1 Answers1

77

At it's base, grok is based on regular expressions, so you can surround a pattern with ()? to make it optional -- for example (%{NUMBER:requestId})?,

If there isn't a grok pattern that suits your needs, you can always create a named extraction like this: (?<version>[\d\.]+) which would extract into version, a string that has any number of digits and dots in it.

Alcanzar
  • 16,985
  • 6
  • 42
  • 59
  • Excellent answer! I have same problem and your answer saved me hours for searching! – Allen Jun 07 '16 at 05:19
  • Yes, this worked perfectly for me too. @Alcanzar I've seen you answer other questions - what good references do you recommend? I find there's a context gap between the grok docs and the underlying reg-exp references! – scipilot Jul 14 '16 at 01:31
  • 2
    Just an observation, that the optional modifier does not seem to apply for the GREEDYDATA pattern. For example: `(%{GREEDYDATA:x})? %{IP:ipaddr} (%{GREEDYDATA:y})?` – daparic Dec 11 '17 at 02:33
  • @ifelsemonkey Facing the same issue. Will post an answer if get any solution or workaround. – Dhairyashil Mar 21 '18 at 15:52
  • 1
    We used the (%{DATA:x})? – Simon LG Aug 29 '18 at 01:25