bash script - awk not processing variables passed as regex

Question

I am writing a bash script. The following awk command does not print anything out even when there is a match. I have copied the awk line and it executes successfully in the interactive shell when I replace $SEARCH_QUERY with real data. This leads me to believe that awk is not translating the data in the variable $SEARCH_QUERY correctly. Any ideas how to fix this or what bash topics I should read up on to understand why it is not working? Thank you

search_contacts()
{
    echo "Enter contact Name:"
    read SEARCH_QUERY
    awk ' /$SEARCH_QUERY/ { print $0 }' contacts.db
}

Don't use all-caps names for your own variables in shell -- such names are used for variables with meaning to the OS or the shell itself, whereas lowercase names are guaranteed not to conflict with the shell's settings or operation. See http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap08.html, fourth paragraph. — Charles Duffy, May 15 '17 at 16:44
As for your original code not working -- the *immediate* issue was use of single quotes (which suppress expansions) and then using an expansion to substitute data into your awk code. However, the *most serious* issue was using string substitution to substitute data into code -- at its core, this is the source of injection vulnerabilities; just like SQL injection, shell injection and awk injection are also things that exist. Because awk can run shell commands, letting someone substitute arbitrary content into an awk script is equivalent to letting them run arbitrary commands on your system. — Charles Duffy, May 15 '17 at 16:46
Thank you Charles. Does using the awk option: search_query prevent the risk of nasty user injected code being executed by awk? — GustavMahler, May 15 '17 at 16:51
You mean the `-v var="$var"` approach? That ensures that the value is strictly treated as data rather than code, so yes, it avoids injection attacks. (`jq` can do the same with `--arg var "$var"`; several other languages/tools let data be passed out-of-band through the environment). — Charles Duffy, May 15 '17 at 16:52
Glad to help. I may actually delete my answer here since there's a good one on the question this is flagged as duplicative of -- you might consider giving the author of that answer an upvote as well. :) — Charles Duffy, May 15 '17 at 17:01

Charles Duffy · Accepted Answer · 2017-05-15T16:51:39.003

In terms of why the original code was broken -- single quotes suppress parameter expansion, so awk was treating $SEARCH_QUERY as a regex itself, ie. looking for a line with the string SEARCH_QUERY after its end -- a regex that can't possibly ever match.

Thus, the naive approach would be to change quoting types, such that your intended substitution actually takes place:

# THIS IS DANGEROUSLY INSECURE; DON'T DO THIS EVER
awk "/$SEARCH_QUERY/" contacts.db

However, as per the comment, the above is dangerously insecure; if your SEARCH_QUERY value ended the regex (included a /), it could then run any arbitrary awk code it wanted. nonmatch/ || system("rm -rf ~") || /anything would do Very Bad Things.

In terms of the best practice -- data should be passed out-of-band from code:

awk -v search_query="$SEARCH_QUERY" '$0 ~ search_query' contacts.db

bash script - awk not processing variables passed as regex

1 Answers1