2

We are using a simple curl to get metrics via an API. The problem is, that the output is fixed in the amount of arguments but not their position within the output.

We need to do this with a "simple" regex since the tool only accepts this.

/"name":"(.*)".*?"memory":(\d+).*?"consumer_utilisation":(\w+|\d+).*?"messages_unacknowledged":(\d+).*?"messages_ready":(\d+).*?"messages":(\d+)/s

It works fine for:

{"name":"queue1","memory":89048,"consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"messages":0}

However if the output order is changed, then it doesn't match any more:

{"name":"queue2","consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"messages":0,"memory":21944}
{"name":"queue3","consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"memory":21944,"messages":0}

I need a relative definition of the strings to match, since I never know at which position they will appear. Its in total 9 different queue-metric-groups.

3limin4t0r
  • 19,353
  • 2
  • 31
  • 52
MA123
  • 21
  • 2
  • 1
    *"Since regular expressions are not fully standardized, all questions with this tag should also include a tag specifying the applicable programming language or tool."* - [regex tag](https://stackoverflow.com/tags/regex/info) – 3limin4t0r Mar 25 '19 at 12:54
  • Why don't you parse the JSON response? – 3limin4t0r Mar 25 '19 at 12:58
  • - its the reguar expression based content Mtach function in Sitescope (IT Monitoring) – MA123 Mar 25 '19 at 13:18
  • - certain restrictions make it complicated to implement an additional parser on the output – MA123 Mar 25 '19 at 13:19

1 Answers1

1

The simple option is to use a regex for each key-value pair instead of one large regex.

/"name":"((?:[^\\"]|\\.)*)"/
/"memory":(\d+)/

This other option is not a regex, but might be sufficient. Instead of using regex, you could simply transform the resulting response before reading it. Since you say "We are using a simple curl" I'm guessing you're talking about the Curl command line tool. You could pipe the result into a simple Perl command.

perl -ne 'use JSON; use Text::CSV qw(csv); $hash = decode_json $_; csv (sep_char=> ";", out => *STDOUT, in => [[$hash->{name}, $hash->{memory}, $hash->{consumer_utilisation}, $hash->{messages_unacknowledged}, $hash->{messages_ready}, $hash->{messages}]]);'

This will keep the order the same, making it easier to use a regex to read out the data.

input

{"name":"queue1","memory":89048,"consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"messages":0}
{"name":"queue2","consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"messages":0,"memory":21944}
{"name":"queue3","consumer_utilisation":null,"messages_unacknowledged":0,"messages_ready":0,"memory":21944,"messages":0}

output

queue1;89048;;0;0;0
queue2;21944;;0;0;0
queue3;21944;;0;0;0

For this to work you need Perl and the packages JSON and Text::CSV installed. On my system they are present in perl, libjson-perl and libtext-csv-perl.

note: I'm currently using ; as separator. If this is included into one of the output will be surrounded by double quotes. "name":"que;ue1" => "que;ue1";89048;;0;0;0 If the value includes both a ; and a " the " will be escaped by placing another one before it. "name":"q\"ue;ue1" => "q""ue;ue1";89048;;0;0;0

3limin4t0r
  • 19,353
  • 2
  • 31
  • 52
  • Just mentioning that we use the curl in a bash script executed on the respective machines to give us the metrics. Could the Perl piping be included in this? – MA123 Mar 25 '19 at 15:02
  • @MA123 Let me just provide it to you here in the comments. Let's say you have a file containing those lines you could simply do `cat response_file.txt | perl -ne '...'` If your `curl` command simply returns text you can do `curl ... | perl -ne '...'`. I'm not familiar with bash scripts, but this is how you'd use it in a normal bash command. – 3limin4t0r Mar 25 '19 at 15:18
  • I'm guessing you could therefore also use it in the script, but you'll have to test that one yourself. However now you mentioned this is in a bash script there might be better options. Have a look at: https://stackoverflow.com/questions/1955505/parsing-json-with-unix-tools – 3limin4t0r Mar 25 '19 at 15:24