11

I am writing a script to read commands from a file and execute a specific command. I want my script to work for either single input arguments or when an argument is a filename which contains the arguments in question.

My code below works except for one problem, it ignores the last line of the file. So, if the file were as follows.

file.txt

file1
file2

The script posted below only runs the command for file.txt

for currentJob in "$@"
do
    if [[ "$currentJob" != *.* ]]  #single file input arg
    then
        echo $currentJob
        serverJobName="$( tr '[A-Z]' '[a-z]' <<< "$currentJob" )"  #Change to lowercase
                                                                   #run cURL job
        curl -o "$currentJob"PisaInterfaces.xml http://www.ebi.ac.uk/msd-srv/pisa/cgi-bin/interfaces.pisa?"$serverJobName"
    else  #file with list of pdbs
        echo "Reading "$currentJob
        while read line; do
            echo "-"$line
            serverJobName="$( tr '[A-Z]' '[a-z]' <<< "$line" )"
            curl -o "$line"PisaInterfaces.xml http://www.ebi.ac.uk/msd-srv/pisa/cgi-bin/interfaces.pisa?"$serverJobName"
        done < "$currentJob"
    fi
done

There is, of course, the obvious work around where after the while loop I repeat the steps for inside the loop to complete those commands with the last file, but this is not desirable as any changes I make inside the while loop must be repeated again outside of the while loop. I have searched around online and could not find anyone asking this precise question. I am sure it is out there, but I have not found it.

The output I get is as follows.

>testScript.sh file.txt
Reading file.txt
-file1
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  642k    0  642k    0     0   349k      0 --:--:--  0:00:01 --:--:--  492k

My bash version is 3.2.48

Ansgar Wiechers
  • 193,178
  • 25
  • 254
  • 328
PhiloEpisteme
  • 857
  • 3
  • 8
  • 19

4 Answers4

26

It sounds like your input file is missing the newline character after its last line. When read encounters this, it sets the variable ($line in this case) to the last line, but then returns an end-of-file error so the loop doesn't execute for that last line. To work around this, you can make the loop execute if read succeeds or it read anything into $line:

...
while read line || [[ -n "$line" ]]; do
    ...

EDIT: the || in the while condition is what's known as a short-circuit boolean -- it tries the first command (read line), and if that succeeds it skips the second ([[ -n "$line" ]]) and goes through the loop (basically, as long as the read succeeds, it runs just like your original script). If the read fails, it checks the second command ([[ -n "$line" ]]) -- if read read anything into $line before hitting the end of file (i.e. if there was an unterminated last line in the file), this'll succeed, so the while condition as a whole is considered to have succeeded, and the loop runs one more time.

After that last unterminated line is processed, it'll run the test again. This time, the read will fail (it's still at the end of file), and since read didn't read anything into $line this time the [[ -n "$line" ]] test will also fail, so the while condition as a whole fails and the loop terminates.

EDIT2: The [[ ]] is a bash conditional expression -- it's not a regular command, but it can be used in place of one. Its primary purpose is to succeed or fail, based on the condition inside it. In this case, the -n test means succeed if the operand ("$line") is NONempty. There's a list of other test conditions here, as well as in the man page for the test command.

Note that a conditional expression in [[ ... ]] is subtly different from a test command [ ... ] -- see BashFAQ #031 for differences and a list of available tests. And they're both different from an arithmetic expression with (( ... )), and completely different from a subshell with( ... )...

Gordon Davisson
  • 118,432
  • 16
  • 123
  • 151
  • 1
    Your suggestion worked, thank you. Based on your answer it seems that at the end of each iteration $line is set to the next value of the next line read and THEN it checks if there are further lines to read with the "read line" argument. The [[ -n "$line" ]] checks if $line is empty? Is this correct. Do you mind explaining exactly what the syntax in the second term means? – PhiloEpisteme Mar 19 '13 at 05:07
  • 1
    Thank you for the description. I am aware of the structure of the logic and the meaning of the || signs. I know c++ but I am not familiar with the -n command. Does it simply mean "is empty"? – PhiloEpisteme Mar 19 '13 at 18:15
1

Your problem seems to be a missing carriage return in your file.

If you cat your file, you need to see the last line successfully appearing before the promopt.

Otherwise try adding :

    echo "Reading "$currentJob
    echo >> $currentJob #add new line
    while read line; do

to force the last line of the file.

Majid Laissi
  • 19,188
  • 19
  • 68
  • 105
0

Using grep with while loop:

while IFS= read -r line; do
  echo "$line"
done < <(grep "" file)

Using grep . instead of grep "" will skip the empty lines.

Note:

  1. Using IFS= keeps any line indentation intact.

  2. You should almost always use the -r option with read.

Jahid
  • 21,542
  • 10
  • 90
  • 108
0

I found some code that reads a file including the last line, and works without the [[ ]] command:

http://www.unix.com/shell-programming-and-scripting/161645-read-file-using-while-loop-not-reading-last-line.html

DONE=false
until $DONE; do
    read s || DONE=true
    # you code
done < FILE
ling
  • 9,545
  • 4
  • 52
  • 49