How to use sed to grep specifc values from a file

Question

I am new to using sed command and my question might probably a basic one.

I am having a log file and I am trying to grep only the errors and extract some required values from the error lines.

I am facing a problem when there are more than 9 pattern groups in sed. I tried using sed -e to have multiple sed but it does not seem to work.

Commands tried

Command #1:

grep ERROR file | sed -n 's_.*key1=$[0-9]*$, key2=$[0-9]*$,.* key3=(.*\), key4=$[a-zA-Z0-9_]*$,.* key5=$[A-Z0-9]*$,.*key6=$[a-zA-Z0-9]*$,.*key7=$[0-9]*$, key8=$.*$, key9=$.*$, key10=$.*$}.*_\1,\2,\3,\4,\5,\6,\7,\8,\9,\10_p'

In the above output, number 10 is being printed instead of the value matching 10th pattern

Command #2:

This AWK command, almost did what I wanted though I am not able to format it as proper comma separated values.

awk -F "\n" '/ERROR/ && match($0,/key1=([0-9]*), key2=([0-9]*),.*key3=(.*),key4=([a-zA-Z0-9_]*),.*key5=([a-z]*).*key6=([a-z]*),.*key7=([A-Z0-9]*),.*key8=([0-9]*).*key9=([a-zA-Z0-9]*)/,a) {
    for (i=1; i in a; i++) {
        printf "%s%s", a[i], (i<length(a) ? OFS : ORS)
    }
    printf "\n"
}' TestFile

What have you tried already? There are plenty of posts talking about this on stackoverflow, and all over the internet. Please let us know what you have found, and tried, and why they didn't help. Try this: http://stackoverflow.com/questions/4318114/circumvent-the-sed-backreference-limit-1-through-9 — stephendl, Apr 07 '17 at 10:10
Possible duplicate of [Circumvent the sed backreference limit \1 through \9](http://stackoverflow.com/questions/4318114/circumvent-the-sed-backreference-limit-1-through-9) — monk, Apr 07 '17 at 10:21
`perl -pe 's/(match)(str)/$2$1/g;'` - this is the suggestion posted in http://stackoverflow.com/questions/4318114/circumvent-the-sed-backreference-limit-1-through-9 but I am still not sure. I have multiple pattern groups in the sed command posted in my question. How could I apply all the pattern in the above perl command — user17, Apr 07 '17 at 10:25
@user17 see the tag wiki: https://stackoverflow.com/tags/sed/info — Sundeep, Apr 07 '17 at 13:22
@user17 sed is for simple substitutions on individual lines, that is all. As such you don't need any materials to learn sed since all it's for is `s/old/new` and for anything else you should use awk. All of the additional sed functionality with arcane language constructs literally became obsolete in the mid-1970s when awk was invented and are used today just for the mental exercise, they should not be used in production code. To learn how to manipulate text, read Effective Awk Programming, 4th Edition, by Arnold Robbins. — Ed Morton, Apr 07 '17 at 16:29

score 1 · Accepted Answer · answered Apr 07 '17 at 16:25

Just use awk. You should already be considering awk since you're using grep+sed. With GNU awk for the 3rd arg to match():

awk -v OFS=',' '/ERROR/ && match($0,/key1=([0-9]*), key2=([0-9]*),.* key3=(.*), key4=([a-zA-Z0-9_]*),.* key5=([A-Z0-9]*),.*key6=([a-zA-Z0-9]*),.*key7=([0-9]*), key8=(.*), key9=(.*), key10=(.*)}/,a) {
    for (i=1; i in a; i++) {
        printf "%s%s", a[i], (i<length(a) ? OFS : ORS)
    }
}' file

but there's probably a much simpler solution which we could help you with if you posted concise, testable sample input and expected output.

How to use sed to grep specifc values from a file

1 Answers1