Get the contents between two given variable strings

Question

How do I get the contents between the 2 given strings? There is no output.

keyword_01=’Serial\s\+Number’;
keyword_02='Signature\s\+Algorithm';
openssl s_client -servername example.com -connect example.com:443 </dev/null 2>/dev/null | openssl x509 -text | grep -i -o -P --color=auto '(?<=$keyword_01).*(?=$keyword_02)'

The basic recommendations work, but when it is pieced as above, it doesn't.

echo "Here is a string" | grep -o -P --color=auto '(?<=Here).*(?=string)'

The objective here is to get the contents of SSL cert contents between 2 fields.

E.g. here, the answer would be '0e:64:c5:fb:c2:36:ad:e1:4b:17:2a:eb:41:c7:8c:b0'

Use `"(?<=$keyword_01).*?(?=$keyword_02)"`, mind the double quotes if you need to interpolate variables and a lazy `*?` quantifier if you need to extract several matches on a single line. I also assume your input string looks like `Serial +Number here Signature +Algorithm`. If you meant to match strings like `Serial Number here Signature Algorithm`, you need to use `keyword_01='Serial\s+Number'; keyword_02='Signature\s+Algorithm';` (that is, remove the escape before `+`). — Wiktor Stribiżew, Jul 11 '18 at 08:53
@Inian It might not be a dupe of only that post, but OP needs to clarify. — Wiktor Stribiżew, Jul 11 '18 at 08:54
@WiktorStribiżew: It seems obvious enough for now. Will revert the close action if they provide more information — Inian, Jul 11 '18 at 08:56
This does not work as well: $ echo $keyword_01 ---- $keyword_02 | grep -i -o -P --color=auto "(?<=$keyword_01).*?(?=$keyword_02)" — Ursa Major, Jul 11 '18 at 09:03
@UrsaMajor See my top comment. What is your input like? Please share a string you need to process. — Wiktor Stribiżew, Jul 11 '18 at 09:13
I understand `0e:64:c5:fb:c2:36:ad:e1:4b:17:2a:eb:41:c7:8c:b0` is the expected output, but what is the *input*? — Wiktor Stribiżew, Jul 11 '18 at 09:27
keyword1='Serial\s\+Number' keyword2='Signature\s\+Algorithm' url=example.com url_with_port=$url:443 openssl s_client -servername $url -connect $url_with_port /dev/null | openssl x509 -text | grep -i -o -P --color=auto '(?<=Serial).*(?=Algorithm)' — Ursa Major, Jul 11 '18 at 19:44
@WiktorStribiżew, it gets input from the url. look at the openssl command. — Ursa Major, Jul 11 '18 at 19:44
@WiktorStribiżew, when it places with openssl, the grep does not work. I think you are close to the answer. — Ursa Major, Jul 11 '18 at 19:49
What is the text you pipe to `grep`? Please add it to the question. We can be "close" all the time, you know, we are "close" matching with `.*`, but to be precise, we should know what the input is like, what the pattern requirements are. Help us to help you. — Wiktor Stribiżew, Jul 11 '18 at 19:55
You can see the input when you remove the last phase which leaves: openssl s_client -servername example.com -connect example.com:443 /dev/null | openssl x509 -text — Ursa Major, Jul 11 '18 at 20:32

hnicke · Accepted Answer · 2018-07-14T04:57:01.637

1

It can be achieved with the builtin bash parameter expansion:

#!/bin/bash  
keyword1='Serial\s\+Number'
keyword2='Signature\s\+Algorithm'

inbetween='0e:64:c5:fb:c2:36:ad:e1:4b:17:2a:eb:41:c7:8c:b0'

src="$keyword1$inbetween$keyword2"

strip_begin=${src#"$keyword1"}
strip_end=${strip_begin%"$keyword2"}

[ "$inbetween" = "$strip_end" ] && echo "It works" || echo "It doesn't work"

For further details, see bash substring removal.

But in order to simply retrieve the two required fields from above command output, try something like this:

certificate=$(openssl s_client -servername example.com -connect example.com:443 </dev/null 2>/dev/null | openssl x509 -text)

serial_number_key="Serial Number:"
serial_number=$(echo "$certificate" | awk "/$serial_number_key/ "'{ getline; print $0}' | tr -d " ")
echo "$serial_number"

signature_algorithm=$(echo "$certificate" | grep -m1 'Signature Algorithm:')
signature_algorithm=${signature_algorithm##*Signature Algorithm: }
echo "$signature_algorithm"

certificate_content=$(awk '/BEGIN CERTIFICATE/ {flag=1;next;} /END CERTIFICATE/ {flag=0} flag' <(echo "$certificate"))
echo "$certificate_content"

The basic idea is to store the certificate in a variable; then use common Unix tools like grep, sed, awk, cut, tr et. al. and shell builtins like parameter expansion to retrieve the individual fields.

There might be a more elegant way if you need to retrieve many more values; but the optimal solution depends on knowing more detailed requirements.

edited Jul 14 '18 at 04:57

answered Jul 11 '18 at 10:41

hnicke

602
4
7

we are almost there, how does this work with: openssl s_client -servername $url -connect $url_with_port /dev/null | openssl x509 -text | grep -i -o -P --color=auto '(?<=Serial).*(?=Algorithm)' – Ursa Major Jul 11 '18 at 19:45
1

Updated my answer to match your specific problem. – hnicke Jul 11 '18 at 20:59
yes, this works for that field. How about the other fields? the reason why I am grep'ing between 2 fields is because they may not be in the same line. Your answer is the closest so far. I am marking this as the main reference. As well, how do you fit in the variable, $keyword_01 into awk '/Serial Number:/ { getline; print $0}' – Ursa Major Jul 11 '18 at 22:13
Can you change "awk '/Serial Number:/ {" to include variable fields instead of hard coded string too? There is a reason why my formal question is on getting between 2 tags, hence I stated to get the contents between two given variable strings, e.g. e.g. get the "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" Thanks. – Ursa Major Jul 12 '18 at 18:15
1

Using variables for pattern matching in awk is, of course, possible -- I've extended the example. It's important to get the quoting right! – hnicke Jul 12 '18 at 21:35
How about to get the contents between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" ? It is the reason why I used 2 tags. Thanks, @hnicke – Ursa Major Jul 14 '18 at 00:56
Added an example. – hnicke Jul 14 '18 at 04:58
error: " -sh: command substitution: line 1: syntax error near unexpected token `(' -sh: command substitution: line 1: `awk '/BEGIN CERTIFICATE/ {flag=1;next;} /END CERTIFICATE/ {flag=0} flag' <(echo "$certificate")' " – Ursa Major Jul 16 '18 at 17:27
could you also make variables on 'BEGIN CERTIFICATE' and 'END CERTIFICATE'? – Ursa Major Jul 16 '18 at 17:27
It should be: awk '/BEGIN CERTIFICATE/ {flag=1;next;} /END CERTIFICATE/ {flag=0} flag' <(echo "$certificate") – hnicke Jul 16 '18 at 21:27
could you also make variables on 'BEGIN CERTIFICATE' and 'END CERTIFICATE'? – Ursa Major Jul 16 '18 at 21:40

Get the contents between two given variable strings

1 Answers1