33

I am very, very new to UNIX programming (running on MacOSX Mountain Lion via Terminal). I've been learning the basics from a bioinformatics and molecular methods course (we've had two classes) where we will eventually be using perl and python for data management purposes. Anyway, we have been tasked with writing a shell script to take data from a group of files and write it to a new file in a format that can be read by a specific program (Migrate-N).

I have gotten a number of functions to do exactly what I need independently when I type them into the command line, but when I put them all together in a script and try to run it I get an error. Here are the details (I apologize for the length):

#! /bin/bash

grep -f Samples.NFCup.txt locus1.fasta > locus1.NFCup.txt
grep -f Samples.NFCup.txt locus2.fasta > locus2.NFCup.txt
grep -f Samples.NFCup.txt locus3.fasta > locus3.NFCup.txt
grep -f Samples.NFCup.txt locus4.fasta > locus4.NFCup.txt
grep -f Samples.NFCup.txt locus5.fasta > locus5.NFCup.txt
grep -f Samples.Salmon.txt locus1.fasta > locus1.Salmon.txt
grep -f Samples.Salmon.txt locus2.fasta > locus2.Salmon.txt
grep -f Samples.Salmon.txt locus3.fasta > locus3.Salmon.txt
grep -f Samples.Salmon.txt locus4.fasta > locus4.Salmon.txt
grep -f Samples.Salmon.txt locus5.fasta > locus5.Salmon.txt
grep -f Samples.Cascades.txt locus1.fasta > locus1.Cascades.txt
grep -f Samples.Cascades.txt locus2.fasta > locus2.Cascades.txt
grep -f Samples.Cascades.txt locus3.fasta > locus3.Cascades.txt
grep -f Samples.Cascades.txt locus4.fasta > locus4.Cascades.txt
grep -f Samples.Cascades.txt locus5.fasta > locus5.Cascades.txt

echo 3 5 Salex_melanopsis > Smelanopsis.mig
echo 656 708 847 1159 779 >> Smelanopsis.mig
echo 154 124 120 74 126 NFCup >> Smelanopsis.mig
cat locus1.NFCup.txt locus2.NFCup.txt locus3.NFCup.txt locus4.NFCup.txt locus5.NFCup.txt >> Smelanopsis.mig
echo 32 30 30 18 38 Salmon River >> Smelanopsis.mig
cat locus1.Salmon.txt locus2.Salmon.txt locus3.Salmon.txt locus4.Salmon.txt locus5.Salmon.txt >> Smelanopsis.mig
echo 56 52 24 29 48 Cascades >> Smelanopsis.mig
cat locus1.Cascades.txt locus2.Cascades.txt locus3.Cascades.txt locus4.Cascades.txt locus5.Cascades.txt >> Smelanopsis.mig

The series of greps are just pulling out DNA sequence data for each site for each locus into new text files. The Samples...txt files have the sample ID numbers for a site, the .fasta files have the sequence information organized by sample ID; the grepping works just fine in command line if I run it individually.

The second group of code creates the actual new file I need to end up with, that ends in .mig. The echo lines are data about counts (basepairs per locus, populations in the analysis, samples per site, etc.) that the program needs information on. The cat lines are to mash together the locus by site data created by all the grepping below the site-specific information dictated in the echo line. You no doubt get the picture.

For creating the shell script I've been starting in Excel so I can easily copy-paste/autofill cells, saving as tab-delimited text, then opening that text file in TextWrangler to remove the tabs before saving as a .sh file (Line breaks: Unix (LF) and Encoding: Unicode (UTF-8)) in the same directory as all the files used in the script. I've tried using chmod +x FILENAME.sh and chmod u+x FILENAME.sh to try to make sure it is executable, but to no avail. Even if I cut the script down to just a single grep line (with the #! /bin/bash first line) I can't get it to work. The process only takes a moment when I type it directly into the command line as none of these files are larger than 160KB and some are significantly smaller. This is what I type in and what I get when I try to run the file (HW is the correct directory)

localhost:HW Mirel$ MigrateNshell.sh
-bash: MigrateNshell.sh: command not found

I've been at this impass for two days now, so any input would be greatly appreciated! Thanks!!

jww
  • 97,681
  • 90
  • 411
  • 885
ChickadeeChick
  • 331
  • 1
  • 3
  • 3
  • 4
    Is the location of `MigrateNshell.sh` in your `PATH`? If trying to run it from the current directory, try `./MigrateNshell.sh`. – Anders Lindahl May 13 '13 at 19:30

12 Answers12

51

For security reasons, the shell will not search the current directory (by default) for an executable. You have to be specific, and tell bash that your script is in the current directory (.):

$ ./MigrateNshell.sh
chepner
  • 497,756
  • 71
  • 530
  • 681
  • 3
    You can also specify the interpreter: `bash MigrateNshell.sh`. This works even without execute permissions. – Otaia May 13 '13 at 20:21
25

Change the first line to the following as pointed out by Marc B

#!/bin/bash

Then mark the script as executable and execute it from the command line

chmod +x MigrateNshell.sh
./MigrateNshell.sh

or simply execute bash from the command line passing in your script as a parameter

/bin/bash MigrateNshell.sh
Ryan Williams
  • 1,465
  • 15
  • 19
18

Make sure you are not using "PATH" as a variable, which will override the existing PATH for environment variables.

janhavi
  • 181
  • 1
  • 3
10

Also try to dos2unix the shell script, because sometimes it has Windows line endings and the shell does not recognize it.

$ dos2unix MigrateNshell.sh

This helps sometimes.

Benjamin W.
  • 46,058
  • 19
  • 106
  • 116
Adolfo
  • 101
  • 1
  • 2
  • This solved it for me. In this case I could see the problem (after this answer suggested it) by noticing in vi the ^M appended to every line. Oddly gvim thought it was already Unix format so it must have just been a single line ending wrong. – lessthanideal May 12 '15 at 16:54
  • I was looking everywhere forgot that I have edited the file in Windows! – Omar Jan 28 '16 at 04:39
3
#! /bin/bash
  ^---

remove the indicated space. The shebang should be

#!/bin/bash
Marc B
  • 356,200
  • 43
  • 426
  • 500
3

Unix has a variable called PATH that is a list of directories where to find commands.

$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/Users/david/bin

If I type a command foo at the command line, my shell will first see if there's an executable command /usr/local/bin/foo. If there is, it will execute /usr/local/bin/foo. If not, it will see if there's an executable command /usr/bin/foo and if not there, it will look to see if /bin/foo exists, etc. until it gets to /Users/david/bin/foo.

If it can't find a command foo in any of those directories, it tell me command not found.

There are several ways I can handle this issue:

  • Use the commandbash foo since foo is a shell script.
  • Include the directory name when you eecute the command like /Users/david/foo or $PWD/foo or just plain ./foo.
  • Change your $PATH variable to add the directory that contains your commands to the PATH.

You can modify $HOME/.bash_profile or $HOME/.profile if .bash_profile doesn't exist. I did that to add in /usr/local/bin which I placed first in my path. This way, I can override the standard commands that are in the OS. For example, I have Ant 1.9.1, but the Mac came with Ant 1.8.4. I put my ant command in /usr/local/bin, so my version of antwill execute first. I also added $HOME/bin to the end of the PATH for my own commands. If I had a file like the one you want to execute, I'll place it in $HOME/bin to execute it.

David W.
  • 105,218
  • 39
  • 216
  • 337
3

Try chmod u+x MigrateNshell.sh

Aminah Nuraini
  • 18,120
  • 8
  • 90
  • 108
2

There have been a few good comments about adding the shebang line to the beginning of the script. I'd like to add a recommendation to use the env command as well, for additional portability.

While #!/bin/bash may be the correct location on your system, that's not universal. Additionally, that may not be the user's preferred bash. #!/usr/bin/env bash will select the first bash found in the path.

broadmonkey
  • 124
  • 2
  • 9
0

Also make sure /bin/bash is the proper location for bash .... if you took that line from an example somewhere it may not match your particular server. If you are specifying an invalid location for bash you're going to have a problem.

James Gawron
  • 871
  • 10
  • 27
0

Add below lines in your .profile path

PATH=$PATH:$HOME/bin:$Dir_where_script_exists

export PATH

Now your script should work without ./

Raj Dagla

Volker E.
  • 5,911
  • 11
  • 47
  • 64
Raj Dagla
  • 1
-1

I'm new to shell scripting too, but I had this same issue. Make sure at the end of your script you have a blank line. Otherwise it won't work.

A Smith
  • 1
-3

First:

chmod 777 ./MigrateNshell.sh

Then:

./MigrateNshell.sh

Or, add your program to a directory recognized in your $PATH variable. Example: Path Variable Example Which will then allow you to call your program without ./

Community
  • 1
  • 1
Bts
  • 107
  • 1
  • 9
  • 7
    Make that `chmod 755`, not `chmod 777`. `777` gives write permission to everyone on the system. `755` gives you full permission, but only read and execute to everyone else. – Keith Thompson Jun 28 '13 at 21:35
  • Whatever you are hoping to accomplish, **`chmod 777` is *wrong* and *dangerous.*** You will want to revert to sane permissions ASAP (for your use case, probably `chmod 755`) and if you have had world writable system files on a public-facing system, at the very least investigate whether it could have been breached and used as a pivot point for breaking into your organization’s network. – tripleee Feb 19 '22 at 09:03
  • changing to 777 could be risky. But still this answer helped me. – mykey May 11 '22 at 14:03