1

While I was running the command directly via the terminal I was getting the desired output. However when I create a script for the same it isn't working the same.

Basically I was replacing all instances of a string in the file using the sed command.

I'm new to the bash scripting and I tried to make all possible changes in the script however I wasn't getting the appropriate results.

What are all the things I need to know to avoid such differences? What am I missing here? Why not just simply copying the command from terminal to the script giving the same behavior?

Below is what I run directly via the terminal:

sed -i 's,'"^${var}"','"$change"',gI' ./filename 
sed -i 's,'" ${var}"','" $change',gI' ./filename

Here is my script:

change="someotherstring"
while read -r line
do
    sed -i 's/'" $line"'/'" $change"'/gI' filename  
    sed -i 's/'"^$line"'/'"$change"'/gI' filename
done < "$2" #file_from_which_i'm_picking_strings_to_replace

While running the command via the terminal I'm able to replace all the possible target strings.

However, on the script the same command is behaving differently and isn't replacing the possible candidates or rather producing abnormal results.

may
  • 11
  • 2
  • Please show the complete code and state the exact problem or error. Also see [How to create a Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve). – jww Aug 10 '19 at 16:35
  • The section "my script" contains my entire script code. I'm picking a string from $2{which is a file given as argument} and replacing all the instances of that in other file "filename". On terminal i just took a string in variable "var" and tried replacing all of its instances in "filename" and it was going good. – may Aug 10 '19 at 16:40
  • Also see [How to use Shellcheck](http://github.com/koalaman/shellcheck), [What is the preferred Bash shebang?](https://stackoverflow.com/q/10376206/608639), [How to debug a bash script?](http://unix.stackexchange.com/q/155551/56041) (U&L.SE), [How to debug a bash script?](http://stackoverflow.com/q/951336/608639) (SO), [How to debug bash script?](http://askubuntu.com/q/21136) (AskU), [Debugging Bash scripts](http://tldp.org/LDP/Bash-Beginners-Guide/html/sect_02_03.html), etc. – jww Aug 10 '19 at 16:47
  • 2
    If I make a guess, the order of your `sed` commands is different in both cases which can be one good reason for different results. If that's not the issue then consider using `—debug` flag with `sed` in order to see the what gets replaced – Mihir Luthra Aug 10 '19 at 17:36
  • I just tried to reproduce your problem and your script works for me. If the answer provided doesn't solve your problem, can you also post a sample input file (the one called `filename` in your script) and a sample file with strings to replace, which together exhibit your problem? And can you add the incorrect output you get with that file pair? – joanis Aug 10 '19 at 18:38

2 Answers2

0

Hmm, I don't see any reason your script should behave differently. I think your problem may be with how it's invoked, or what the current directory is when it's invoked. A couple of suggestions, though.

You should mention the shell specifically in your script, and we can simplify the quoting: 

#!/bin/sh
change=someotherstring
while read -r line
do
    sed -i "s/ $line/ $change/gI" filename  
    sed -i "s/^$line/$change/gI" filename
done < "$2" # file of strings to replace

I don't see what 

sed -i "s/ $line/ $change/gI" filename

can do. You just read in $line; there can't be any leading space in front of its first character. So only the second line is effective. (If you want to use several sed commands, you can use its -e option more than once, or separate the commands in a single string with ";".)

If you wordlist is long, you'll find it's more efficient to use sed to generate sed commands first, and then use that file against all your targets at once. Roughly: 

#! /bin/sh
set -e
change="$1"
words="$2"
shift 2
sed -E "s:.+:s/&/$change/:" "$words" > "$words.sed"
sed -E -f "$words.sed" $@

That also has the benefit of manifesting your sed commands to a file, where you can inspect them before (or after) executing them. Otherwise, you have to be very very sure your shell transformation.

I habitually use set -e as line #2 of my shell scripts. If an error is encountered, I want it to stop unless handled explicitly.

Finally, don't forget you can use sh -x scriptname to see how your script is being evaluated.

James K. Lowden
  • 7,574
  • 1
  • 16
  • 31
  • Actually, if you see the terminal commands .. the first one is to check whether the candidate string is the first thing in the line.. and if it is .. change it.. and second command was checking ... whether prior to the candidate string, is there any blank space(to check it's not a substring/suffix of any other string)..if there's any blank space .. change it. so, i just wrote them in the script. – may Aug 10 '19 at 18:07
  • You should s/read -r/IFS= read -r/ and s/\$@/"$@"/ in your scripts. – fumiyas Aug 12 '19 at 09:41
0

I had a problem with similar behavior to the title and came across this question in my search for an answer.

In the event a similar under-experienced individual ends up here, because they don't know how else to describe and search for their issue, I'll share my experience and point those readers in a direction that may assist their troubleshooting.

The abnormal behavior I experienced was comparing the results from executing commands in an ssh terminal versus executing a script remotely via ssh. Ultimately two factors came into play.

  1. The $PATH variable is set differently in these two instances. An ssh terminal is an interactive shell and the remotely executed script is a non-interactive shell. Additionally there is also a difference between a login shell and a non-login shell.

My understanding of all the nuances between the combinations of these is minimal as this is my first exposure to them, however I ended up having to add locations to the PATH to have my script run as the commands did in the interactive shell.

  1. Variable assignments. $PATH in a remotely executed script is populated by the PATH of the local machine, my laptop. In order to use $PATH or any variable on the remote machine, the $ in the script requires escaping like this, \$PATH.

Because the $PATH of my local machine and the $PATH in my interactive terminal window were so similar, it took me a long time to realize that I was not seeing the \$PATH values in my script that I thought I was. This oversight was compounded since I also didn't realize the assignment of $VARiables throughout my script were not \$VARiables. This resulted in very abnormal and unexpected results, which brought me to this page.

Although not an explicit solution for the original problem, I am hoping this will help others lured here by the ambiguous nature of the title.

B H
  • 111
  • 1
  • 3