if condition on while iteration loop bash

Question

I have a file with contents like below;

864440 17:59:48.143
864440 17:59:48.221
864441 17:59:48.159
864441 17:59:48.221
864442 17:59:48.159
864442 17:59:48.237
864443 17:59:48.174
864444 17:59:48.190
864444 17:59:48.253
864445 17:59:48.206
864445 17:59:48.268
864446 17:59:48.221
864446 17:59:48.284

I am doing a while loop to read the 1st and 2nd value and put it on a variable like below;

while read list; do
     jbnum=`echo $list|awk '{print $1}'`
     time=`echo $list|awk '{print $2}'`
done < jobstmp.txt

I would like to add an if condition that if the jbnum is equal to the next iteration it would get the time difference of the same job number if not then do nothing and go to next iteration

Thanks for all those who answered the solution below based on the code provided by Chris Maes

while read jobnum time
do
    if [[ "$prevjobnum" == "$jobnum" ]]
    then
        ENDTIME=`date +%s.%N -d "$time"`
        STARTTIME=`date +%s.%N -d "$prevtime"`
        DIFF=$(echo $ENDTIME - $STARTTIME | bc | sed 's/0\{1,\}$//')

        echo "$prevjobnum time is $DIFF seconds"
    fi
    # keep variables for next iteration
    prevjobnum=$jobnum
    prevtime=$time
done < jobstmp.txt

so for example you want to perform an action in the first line, since 864440 occurs later on? Note that going backwards when reading a file is not possible, so you should clarify what exactly you want to do, because probably you can do it on the next line, when noticing the field is repeated. — fedorqui, Aug 30 '16 at 09:52
@fedorqui i have updated the question to indicate exactly what i would like to do if the value of the 1st column is the same on the next iteration — Dren, Aug 30 '16 at 09:57
Thanks! To make your question complete, provide also what would be the desired output for this given input. — fedorqui, Aug 30 '16 at 09:58

Chris Maes · Accepted Answer · 2016-08-30T09:49:43.333

you cannot watch ahead with read. You'll have to use some intermediate variables; something like (note that you can immediately read multiple variables):

while read jobnum time
do
    if [[ "$prevjobnum" == "$jobnum" ]]
    then
        #do your stuff with the variables $prevjobnum and $prevtime. for example:
        echo "prevjobnum=$prevjobnum , prevtime=$prevtime"
    fi
    # keep variables for next iteration
    prevjobnum=$jobnum
    prevtime=$time
done < jobstmp.txt

this gives:

prevjobnum=864440 , prevtime=17:59:48.143
prevjobnum=864441 , prevtime=17:59:48.159
prevjobnum=864442 , prevtime=17:59:48.159
prevjobnum=864444 , prevtime=17:59:48.190
prevjobnum=864445 , prevtime=17:59:48.206
prevjobnum=864446 , prevtime=17:59:48.221

Many thanks. I have updated my question with the solution I am looking for based on your recommendations. Cheer mate. — Dren, Aug 30 '16 at 11:51

score 0 · Answer 2 · edited May 23 '17 at 12:14

Note that your approach can be improved in a way that you don't have to parse the $list. Use read var1 var2 var3... to catch them individually as described in Read a file line by line assigning the value to a variable:

This way, you can have a code like this:

while IFS= read -r jbnum time; do
    if [ "$prev_jbnum" == "$jbnum" ]; then
         echo "they are equal!"
    fi
    prev_jbnum=$jbnum
done < jobstmp.txt

However, to compare one column with the previous one you can use awk in a more simpler way:

$ awk '$1==prev {print $0, "equal!"} {prev=$1}' file
864440 17:59:48.221 equal!
864441 17:59:48.221 equal!
864442 17:59:48.237 equal!
864444 17:59:48.253 equal!
864445 17:59:48.268 equal!
864446 17:59:48.284 equal!

To compare the times when the first field matches, you can use something like mktime(), only that it does not handle milliseconds.

Also, you have to do some massaging to the data to be able to use this function:

{
 gsub(/[^0-9]/, " ", $2)                # replace every : and . with spaces
 current_time=mktime("1970 01 01 " $2); # generate a date prepending 1/1/1970
}

you awk command isn't printing the previous lines as supposed... first line should be "864440 17:59:48.143" — Chris Maes, Aug 30 '16 at 09:49
@ChrisMaes I am printing the line that occurs when the first column is repeated. I will ask some clarification to the OP, since it is not very clear what he is looking for. — fedorqui, Aug 30 '16 at 09:51
I agree that his question is not well written, but from his last line; I deduce he wants the first line. — Chris Maes, Aug 30 '16 at 09:55

Aaron · Answer 3 · 2016-08-30T09:56:47.710

If your ids are always 6 chars wide, you could use uniq to filter the first lines with a specific first field value:

$ echo """864440 17:59:48.143
864440 17:59:48.221
864441 17:59:48.159
864441 17:59:48.221
864442 17:59:48.159
864442 17:59:48.237
864443 17:59:48.174
864444 17:59:48.190
864444 17:59:48.253
864445 17:59:48.206
864445 17:59:48.268
864446 17:59:48.221
864446 17:59:48.284""" | uniq -w 6

864440 17:59:48.143
864441 17:59:48.159
864442 17:59:48.159
864443 17:59:48.174
864444 17:59:48.190
864445 17:59:48.206
864446 17:59:48.221

You could then iterate on the output lines to make actions that should only be done once by first field value.

Note that the data has to be sorted for uniq to work properly.

doesn't look like the first occurrence to me... first line should be "864440 17:59:48.143" — Chris Maes, Aug 30 '16 at 09:48

Paradise228 · Answer 4 · 2016-08-30T09:52:04.950

0

I would suggest you to do a first read before your loop

read line

while read nextLine; do
    jbnum=`echo $line|awk '{print $1}'`
    time=`echo $line|awk '{print $2}'`

    nextjbnum=`echo $nextLine|awk '{print $1}'`
    nexttime=`echo $nextLine|awk '{print $2}'`

    if [jbnum == nextjbum]; then
       DO STUFF ....
    fi

    #Then save the current line before processing the next
    line=nextLine
done

edited Aug 30 '16 at 09:52

answered Aug 30 '16 at 09:49

Paradise228

717
3
16

good idea; but you are using an undefined variable $list; and you are launching many subprocesses which isn't very optimal. – Chris Maes Aug 30 '16 at 09:51
That's the problem with bad copy/pasting :-) And yeah you may be right about the optimization – Paradise228 Aug 30 '16 at 09:53

score 0 · Answer 5 · answered Aug 30 '16 at 11:51

Perform your calculation inside the {} that contains

print $1,"//do your calculation here"

awk 'idMap[$1]==""{idMap[$1]=$2;print;next;}{print $1,"//do your calculation here"}' <filaname>

The generated output looks like this at the moment.

864440 17:59:48.143
864440 //do your calculation here
864441 17:59:48.159
864441 //do your calculation here
864442 17:59:48.159
864442 //do your calculation here
864443 17:59:48.174
864444 17:59:48.190
864444 //do your calculation here
864445 17:59:48.206
864445 //do your calculation here
864446 17:59:48.221
864446 //do your calculation here

if condition on while iteration loop bash

5 Answers5

Linked