Search after a match once per line

Question

I'm trying to create a script to get all the Apache response after specific word "statusDescription", but I have a problem that I got the output duplicated for some rows as matched word or response maybe written 2 times @ the same line

Sample of log "1 line":

GET/en?status=1&newMainBalance=5486&serviceAmount=700&ExternalTrxId=asdf&PgTrxId=tfpsadf&amount=0&statusDescription=Failed&customerCode=1.1&newDedicatedBalance=0&secureHash=56a7sdyf&paidAmount=1000&responseMsg=%a1%a1%A1(PG_ID)&language=enHTTP/1.1"200186243**1/1210669**1"-""-""-""https://example.com.eg?statusDescription=Failed&externalTrxId=123&status=203&secureHash=asdf&pgTrxId=asdf

i tried the below command to get any match between "statusDescription" and "&"

cat test.txt  | perl -nle'print $1 while /statusDescription(.*?)\&/g'
cat test.txt  | perl -nle'print $1 while /statusDescription(.*?)\&/g'

output:

=Failed
=Failed

i except the result to be 1 line only like

=Failed

or

=Failed=Failed

Also watch the [useless use of `cat`](/questions/11710552/useless-use-of-cat) — tripleee, Apr 01 '19 at 14:21

ikegami · Answer 1 · 2019-04-01T13:53:04.503

The -l is having the effect of printing a line feed after each print.

The following will print the value of the (first) statusDescription field of the requested URL (while ignoring the referrer URL). It will even correctly decode the value for you (if it contains escaped characters such as + or %20).

perl -MURI -MURI::QueryParam -nle'
   my ($request_url) = /^\S+\s+(\S+)/
      or next;
   $request_url = URI->new($request_url, "http");
   my $status = $request_url->query_param("statusDescription")
      or next;
   print $status;
' test.txt

Stefan Becker · Accepted Answer · 2019-04-03T13:01:26.893

1

You used -l option without a value. To quote from perlrun:

If octnum is omitted, sets $\ to the current value of $/

which by default is \n. Hence every print will automatically get a \n appended.

$  echo 'Match=once& should only Match=twice& once' | perl -nle 'print "MATCH ->$1<-" while /Match(.*?)&/g'
MATCH ->=once<-
MATCH ->=twice<-

# the same without -l
$ echo 'Match=once& should only Match=twice& once' | perl -ne 'print "MATCH ->$1<-" while /Match(.*?)&/g'
MATCH ->=once<-MATCH ->=twice<-

# if your post-processing requires matches from different lines to appear
# on separate lines then append a conditional print "\n" at the end
$ echo 'Match=once& should only Match=twice& once' | perl -ne 'print "MATCH ->$1<-" while /Match(.*?)&/g; print "\n" if $1'
MATCH ->=once<-MATCH ->=twice<-
$ echo 'nomatch=once& should only nomatch=twice& once' | perl -ne 'print "MATCH ->$1<-" while /Match(.*?)&/g; print "\n" if defined $1'
$

For $/ (input record separator) and $\ (output record separator) see also perlvar.

edited Apr 03 '19 at 13:01

answered Apr 01 '19 at 12:58

Stefan Becker

5,695
9
20
30

thanks a lot for your support , can you please help me in 1 more thing. it worked with me , but i faced 1 more issue that it return null when Statusdescription is located at the end of line . for example : GET/en?status=1&newMainBalance=5486&serviceAmount=700&ExternalTrxId=asdf&PgTrxId=tfpsadf&amount=0&statusDescription=Failed" so it doesn't match the criteria for "&" can i make it more generic , just match 1 word after "StatusDescription=" and before "&" or not – Ahmed Farouk Apr 03 '19 at 12:54
That's actually another question, but you could try to split lines on `&` and then match on the parts. E.g. `perl -ne 'foreach my $p (split(/&/)) { print "$1\n" if $p =~ /^statusDescription(=.*)/; }'` – Stefan Becker Apr 03 '19 at 13:15
but if i split the line on "&" , it will duplicate the output in case i have 2 statusdescription on the same line (one before & and another one in end of the line) and i need to terminate after first match to get accurate result thanks alot for your support in advance , you really helped me alot – Ahmed Farouk Apr 03 '19 at 14:02
OK, you only want to find and print the first match? Then this should do it `perl -ne 'foreach my $p (split(/&/)) { if ($p =~ /^statusDescription(=.*)/) { print "$1\n"; last } }'` – Stefan Becker Apr 03 '19 at 16:02
Hello Stefan Becker i faced a new issue that i found duplicates some time so i need to add a new match to filter with it so i need to get statusdescrition as before and add pgtrxid to be like that (pgTrxId=asdf&statusDescription=Failed) bearing in mind to ignore case sensitive for pgtrxid – Ahmed Farouk Apr 15 '19 at 08:35

Search after a match once per line

2 Answers2