1

Here's my problem, I have written the following line of code to format properly a list of files found recursively in a directory. 

find * | sed -e '/\(.*\..*\)/ !d' | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/"

The second step is to write the output of this command in a script. While the code above has the expected behavior, the problem occurs when I try to store its output to a variable, I get a bad substitution error from the first sed command in the line.

#!/bin/bash
nsisscript=myscript.sh
FILES=*
for f in $(find $FILES); do 
v=`echo $f | sed -e '/\(.*\..*\)/ !d' | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/"`
sed -i.backup -e "s/\;Insert files here/$v\\n&/" $nsisscript 
done

Could you please help me understand what the difference is between the two cases and why I get this error ?

Thanks in advance!

Patrick
  • 63
  • 1
  • 3
  • 10
  • Hum... I guess I found the solution myself. Inside the `` delimiters, the "$" is being interpreted for some reason whether alone or with one backslash before it. The solution was to have 2 backslashes before each "$" character. v=`echo $f | sed -e '/\(.*\..*\)/ !d' | sed -e "s/^.*/\\${File} \\${INST\_FILES} &/" ..... I'd be happy to hear why exactly this happens for this specific character though. – Patrick Aug 23 '11 at 08:20
  • 1
    Not related to your problem per se, but it seems you do not know about it being possible to change sed's delimiters so you do not have to escape slashes. Just choose another character which you do not need, such as `:`: `sed "s:pattern:substitution:`. It will greatly improve readability on regexes treating paths – carlpett Aug 23 '11 at 12:16
  • [BashPitfalls #1](https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29) is relevant. Likewise, [DontReadLinesWithFor](https://mywiki.wooledge.org/DontReadLinesWithFor) – Charles Duffy Mar 20 '23 at 15:17
  • Also, `echo $f` is itself buggy -- see [I just assigned a variable, but `echo $variable` shows something different!](https://stackoverflow.com/questions/29378566/i-just-assigned-a-variable-but-echo-variable-shows-something-else) – Charles Duffy Mar 20 '23 at 15:24
  • And `FILES=*` is literally exactly the same as `FILES='*'` -- it's assigning the string, not expanding a wildcard. I'd generally recommend avoiding it altogether: `find .` is better-behaved, because even if you have a file starting with a `-`, its name will become `./-`, which doesn't cause other tools to misbehave when passed as an argument. – Charles Duffy Mar 20 '23 at 15:25
  • But yes, the real problem is all about backticks, and they shouldn't be used anyhow in modern times; there's a good reason `$(...)` has been standardized since the early 90s, so backticks are still around only for compatibility with scripts written in the 70s and 80s; you shouldn't be writing anything that uses them today. – Charles Duffy Mar 20 '23 at 15:26

2 Answers2

1

Parsing inside of backquote-style command substitution is a bit weird -- it requires an extra level of escaping (i.e. backslashes) to control when expansions take place. Ugly solution: add more backslashes. Better solution: use $() instead of backquotes -- it does the same thing, but without the weird parsing and escaping issues.

BTW, your script seems to have some other issues. First, I don't know about the sed on your system, but the versions I'm familiar with don't interpret \n in the substitution as a newline (which I presume you want), but as a literal n character. One solution is to include a literal newline in the substitution (preceded by a backslash).

Also, the loop executes for each found file, but for files that don't have a period in the name, the first sed command removes them, $v is empty, and you add a blank line to myscript.sh. You should either put the filtering sed call in the for statement, or add it as a filter to the find command.

#!/bin/bash
nsisscript=myscript.sh
nl=$'\n'
FILES=*
for f in $(find $FILES -name "*.*"); do 
    v=$(echo $f | sed -e "s/^.*/\${File} \${INST\_FILES} &/" | sed -e "s/\( \)\([a-zA-Z0-9]*\/\)/\/\2/" | sed -e "s/\(\/\)\([a-zA-Z0-9\_\-\(\)\{\}\$]*\.[a-zA-Z0-9]*\)/ \2/")
    sed -i.backup -e "s/\;Insert files here/$v\\$nl&/" $nsisscript 
done
Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151
  • Hi, thanks for the answer. I like the $() trick instead of back quotes which I think are cumbersome. On the other hand, I didn't have any issue with the newline, each time a line was entered in the file, it was properly followed by a newline and the ";Insert files here" underneath as expected. – Patrick Aug 25 '11 at 07:20
0

Well my guess was that your escaping of underscore in INST_FILES is strange as underscore is not a special character in shell nor in sed. The error disappear when you delete the '\' before '_'

my 2 cents

neuro
  • 14,948
  • 3
  • 36
  • 59
  • You are right to say that the error disappears. On the other hand the behavior changes and the output is not what I expected anymore. The double backslash solution seems to work though. – Patrick Aug 23 '11 at 08:24