1

I have a command I need to run in a bash script that takes the output of a previous command and inserts it into a new command.

So for example: VARIABLE=$(cat "file.txt" | grep "text")

Then use that variable in another command:

mycommand -p "$VARIABLE"

The catch is the $VARIABLE will always contain special characters, namely $ and / which I need so I need to single quote that so the special characters are taken literal.

I've tried \""$VARIABLE"\" which didn't work.

What I'm trying to accomplish is I need to grab a line out of a text file that includes my search term, which is unique and only one line will have it.

I then need to input that line (well, half of it, I'm also cutting the line and using the second half) into another command. I am able to successfully grab the text I need which I verified by echoing the variable afterwards. The problem is the variable contains $ and \ that are being interpreted as special characters.

For example:

my command -p $345$randomtext\.moretext

Without single quoting the variable it is interpreted which throws errors.

jcalton88
  • 160
  • 1
  • 12
  • I'll try, working off mobile and it didn't format correctly. – jcalton88 Oct 31 '16 at 05:40
  • Does `printf "%s\n" my command -p "$VARIABLE"` show that the special characters are being interpreted? – muru Oct 31 '16 at 05:49
  • 2
    The premise of your question is wrong. If `VARIABLE` is set to `$345$randomtext\.moretext`, then `my command -p "$VARIABLE"` is equivalent to `my command -p '$345$randomtext\.moretext'` (as desired), *not* to `my command -p $345$randomtext\.moretext`. So the problem must be somewhere inside the definition of `my command`. – ruakh Oct 31 '16 at 05:49
  • That may be, it echoes without any quotes. I was assuming that is how it was then interpreted. If I take the echoed command and run it manually it still fails, if I enclose the $345$randomtext\.moretext in single quotes it runs successfully. – jcalton88 Oct 31 '16 at 05:56
  • @jcalton88 `echo` can be very misleading, since it prints its arguments *after* the shell has interpreted and removed quotes, escapes, etc. Try `echo my command -p '$345$randomtext\.moretext'` -- you'll see that it doesn't include the quotes in the output, exactly the same as `my command -p "$VARIABLE"`. You can use `set -x` to get the shell to print the simplified equivalent of each command as it executes it to get a better idea what's really going on. – Gordon Davisson Oct 31 '16 at 06:02
  • I will do that and update tomorrow with my findings. Thanks for the suggestions – jcalton88 Oct 31 '16 at 06:07
  • Since manually typing the command using single quotes works, I'm going to guess that there are some unexpected characters in the file. These characters (perhaps "\r" ?) are stored in VARIABLE, but are not present when you type the text manually. To verify, do the following: `grep "text" file.txt|xxd` and check the hex dump for any unexpected values. – Eric Nov 01 '16 at 02:15

1 Answers1

3

If should work as you have written it here (assuming that I'm parsing the garbled formatting correctly):

VARIABLE=`cat "file.txt" | grep "text"`
mycommand -p "$VARIABLE"

If there are problems handling special characters in the search results (other than null, which cannot be stored in a shell variable), then they're almost certainly due to mishandling in mycommand, not in how it's being called. Here's an example:

$ cat file.txt 
There are some $special \characters\\\\ in this text file.
This is another line of text.
$text $VARIABLE $HOME etc.
$ VARIABLE=`cat "file.txt" | grep "text"`
$ printargs -p "$VARIABLE"
Got 2 arguments:
    '-p'
    'There are some $special \\characters\\\\\\\\ in this text file.\nThis is another line of text.\n$text $VARIABLE $HOME etc.'

printargs is a simple python program that prints a pythonic representation of its arguments (note that it doubled all of the backslashes, and printed the line breaks as '\n', because that's how you'd write them in a python string):

#!/usr/bin/python

import sys

print "Got", len(sys.argv)-1, "arguments:"
for arg in sys.argv[1:]:
    print "    " + repr(arg)

BTW, I have a couple of stylistic suggestions: use lowercase names for shell variables to avoid conflicts with environment variables that have special meanings (e.g. assigning to $PATH will lead to trouble). Also, cat file | grep pattern is a useless use of cat; just use grep pattern file.

Community
  • 1
  • 1
Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151
  • It may be in mycommand (which albeit I called it my command it's not, it's a built in command for another program). If I run the subsequent command manually as `mycommand -p '$345$randomtext\.moretext'` it successfully runs. I assumed if that was the case it was the shell botching the interpretation, not mycommand. – jcalton88 Oct 31 '16 at 06:01
  • I also agree with the variable naming, I will adjust that. As for the use of cat, it's been a long time since I've written scripts and I only ever use grep when checking if something is running from ps aux and always pipe that into grep. I guess the habit carried over and I forgot or didn't realize I could use grep like that. Thanks! – jcalton88 Oct 31 '16 at 06:04
  • I'm not sure what I did differently today vs yesterday but just enclosing the variable in double quotes worked. I did the same thing yesterday but during my troubleshooting I must've changed something else, possibly how I was getting the variable. Anyway, this is correct so I will select this as the answer. – jcalton88 Nov 01 '16 at 02:20