3

I have a .txt file I am parsing that has multiple values for one field, inputFields. 

INPUT
@07  
@09
@11

I can successfully grok, mutate, and split inputFields into an array containing all the values.

pattern.txt

INPUT_FIELDS (?<inputFields>INPUT\s*(@\d*)*)

logstash.conf

grok {
        match => [ "message", "%{INPUT_FIELDS}"]
    }
mutate {
        gsub => ["inputFields", "INPUT\s*@", ""]
        split => ["inputFields", "@"]
    }

Gives me this output

"inputFields" => [
  [0] "07",
  [1] "09",
  [2] "11"
 ]

The problem is, I want to use the translate plugin, which does not work on array values.

I would like to split this array into multiple fields in the same document. Preferably, each field would be labeled "input_field[i].

For example:

"input_field1" => "07",
"input_field2" => "09", 
"input_field3" => "11"

I have tried tailoring this response Logstash grok filter - name fields dynamically, which uses Ruby, to fit my needs, but it splits the fields into multiple documents, or keeps them in the same field without splitting.

Could someone lend me a hand with the Ruby code, or split plugin config?

Update I was able to create separate events for the input fields, but I am unable to create the fields with an index. I must use the field itself to dynamically create a new field.

I removed the mutate { split }, and split it with Ruby instead.

Here is my new code

ruby {
        code => "
            inputs = event['inputFields'].split('@')
            for input in inputs
                event['inputField_' + input] = input
            end
        "
    }

This creates 

"inputField_07" => "07"
"inputField_09" => "09"
"inputfield_11" => "11"

I would still take any advice about how to change these to inputField_[index]. I tried using different variables in Ruby, but I cannot add them to the new field name.

Dynamically naming the fields has created a new problem. I would like to use the translate plugin, but you must select a specific field for translate to look at. Since the fields I created all have different names, I cannot point to any of them. I have tried using regular expressions to match the field, but it is not supported by translate.

Suggestions??

Mattatat-tat
  • 459
  • 1
  • 7
  • 9

1 Answers1

3

You could achieve your goal of splitting into new fields without ruby code, in this way:

mutate {
    # split the field on ::
    split => ["api_class" , "::"]
    # save the last element of the array as the api_method.
    add_field => ["api_method", "%{[api_class][-1]}" ]
}

I found this answer myself on this page.

Good luck!

rocambille
  • 15,398
  • 12
  • 50
  • 68
André van der Heijden
  • 111
  • 1
  • 9