grok filter (regex) to extract string within square brackets

Question

My application log entries are given below:

2015-06-24 14:03:16.7288  Sent request message [649b85fa-bfa0-4cb4-8c38-1aeacd1cbf74] <Request>sometext</Request>

2015-06-24 14:38:05.2460  Received response message [649b85fa-bfa0-4cb4-8c38-1aeacd1cbf74] <Response>sometext</Response>

I am using logstash grok filter to extract the xml content and the client token with the square bracket.

grok {  
    match => ["message", "(?<content>(<Request(.)*?</Request>))"]   
    match => ["message", "(?<clienttoken>(Sent request message \[(.)*?\]))"]
    add_tag => "Request"
    break_on_match => false
    tag_on_failure => [ ]
}

grok {  
    match => ["message", "(?<content>(<Response(.)*?</Response>))"] 
    match => ["message", "(?<clienttoken>(Received response message \[(.)*?\]))"]
    add_tag => "Response"
    break_on_match => false
    tag_on_failure => [ ]
}

Now the result looks like below

For the first log line:

Content =  <Request>sometext</Request>
clienttoken = Sent request message [649b85fa-bfa0-4cb4-8c38-1aeacd1cbf74]

For the second log line:

Content = <Response>sometext</Response>
clienttoken = Received response message [649b85fa-bfa0-4cb4-8c38-1aeacd1cbf74]

But I want the result to be like this:

Content = <Request>sometext</Request>
clienttoken = 649b85fa-bfa0-4cb4-8c38-1aeacd1cbf74

Please let me know how to extract only the strings within the square bracket without all the matching string in the pattern.

is there any way to print group index 1? `"(?(Received response message \[(.*?)\]))"` — Avinash Raj, Jun 25 '15 at 04:42

score 3 · Accepted Answer · answered Jun 25 '15 at 04:45

3

You may use lookbehind and lookahead assertions.

(?<=Sent request message \[).*?(?=\])

likewise do the same for response message.

answered Jun 25 '15 at 04:45

Avinash Raj

172,303
28
230
274

1

Does anyone have an example of the full grok syntax for matching this (with the field name)? I seem to be having issues nesting my lookahead/lookbehind within my custom match pattern. ie `(?(?<=lookbehindregex).+?(?=lookaheadregex))` - Is this the correct approach? – Toby Apr 10 '19 at 15:50
1

@Toby did you figure it out? Facing the same issue. My lookahead and lookbehind work in REGEX tools, but with logstash they fail – AgentX May 31 '19 at 09:39
@AgentX I did not. [I asked the question here on Stack Overflow](https://stackoverflow.com/questions/55616142/syntax-for-lookahead-and-lookbehind-in-grok-custom-pattern/55635937) and did not receive an answer. – Toby Jun 04 '19 at 01:58

grok filter (regex) to extract string within square brackets

1 Answers1