2

I would like to pre-process following log structure with nxlog and send it then to graylog.

My custom app log structure:

timestamp;field1;field2; ---- Start of good event ----
timestamp;field3;field4;field5;field6
timestamp;field7;field8;field9;field10
timestamp;field11;field12; --- End of good event ---
timestamp;FAIL;field13;field14
timestamp;FAIL;field15;field16

The GELF output from nxlog should contain full_message with "good event" or "bad event".

"good event" = 1 line as follows:

timestamp;field1;field2; ---- Start of good event ----;timestamp;field3;field4;field5;field6;timestamp;field7;field8;field9;field10;timestamp;field11;field12; --- End of good event ---

"bad event" should contain 1 line as follows:

timestamp;FAIL;field13;field14; timestamp;FAIL;field15;field16

I have no problem to parse "good event" with xm_multiline and define it's HeaderLine and EndLine.

But I have absolutely no idea, how to parse two different multilines. Could you give me any hint, please?

Is it possible to use if-else statement with "InputType"? I mean "if condition1 then InputType good-event and some-actions else InputType bad-event and some-actions". Or it needs totally different approach - e.g. no xm_multiline usage but some kind of regex magic?

Thanks in advance.

nimmie
  • 21
  • 1

1 Answers1

0

You can still use xm_multiline. You just need to define the two different patterns with regex.

Since you didn't provide your configuration I'll use my configuration for a different log format as an example.

I have a java application I need to monitor the logs for that doesn't use consistent time formatting, so messages might look like this:

2019-04-24 00:00:13,952 WARN  [SemaphoreArrayListManagedConnectionPool] (QuartzScheduler_quartzScheduler-wildflyapp0201401_ClusterManager) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
 new connection: null: javax.resource.ResourceException: IJ031084: Unable to create connection
        at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createLocalManagedConnection(LocalManagedConnectionFactory.java:336)
        at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:343)

Or like this:

14:00:34,426 INFO  [stdout] (default task-73) com.xyz.england.idserver.comp.impl.Service DEBUG  [Get][db113034-ecc6-4c0d-86f2-moo3e33942f2] Job Package id.
14:00:34,426 INFO  [stdout] (default task-73) [DEBUG 2019-04-24 14:00:34,426]  [Get][db113034-ecc6-4c0d-86f2-moo3e33942f2] Job Package id.
14:00:34,427 INFO  [stdout] (default task-39) com.xyz.england.idserver.comp.impl.Service DEBUG  [Get][0c4d63c0-74d7-4599-bc40-mooa84cf62ea] Job Package id.
14:00:34,427 INFO  [stdout] (default task-39) [DEBUG 2019-04-24 14:00:34,425]  [Get][0c4d63c0-74d7-4599-bc40-mooa84cf62ea] Job Package id.

If the log used one or the other time formats I could have used one of these two configurations:

<Extension java_multiline>
    Module          xm_multiline
    HeaderLine      /^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d /
</Extension>

OR

<Extension java_multiline>
    Module          xm_multiline
    HeaderLine      /^\d\d:\d\d:\d\d,\d\d\d/
</Extension>

Since that wasn't the case I had to include them in a single statement using alternation, specifically using the pipe symbol aka the OR operand:

<Extension java_multiline>
    Module          xm_multiline
    HeaderLine      /^\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d,\d\d\d |^\d\d:\d\d:\d\d,\d\d\d /
</Extension>

Using this regex statement either time format will match as my header line.

Necco
  • 1
  • 1