while loop stops before false

Question

Trying to zip up all logs except for one in a directory. I know there are other ways to accomplish what I'm trying to do, but I'm mainly curious as to WHY the following is happening.

I have the following script:

#!/usr/bin/env bash

TOTAL=$(ls -lah ./*.log | wc -l)
let "TOTAL--"

COUNT=0
while [[ $COUNT < $TOTAL ]]; do
  gzip $(ls -1 | grep -v '.gz' | grep '.log' | head -n 1)
  let "COUNT++"
  echo "$TOTAL - $COUNT = $(($TOTAL-$COUNT))"
  sleep 2
done

To set up the environment for this, you could do something like:

cd
mkdir tmp0
cd tmp0
touch test_{1..20}.log

Then put this script inside the same directory (yes, yes you could also just set a var to reference the directory too). Anyways, when set up like this and run, it stops while $COUNT is still less than $TOTAL:

[ec2-user@ip-172-31-0-70 ~]$ cd
[ec2-user@ip-172-31-0-70 ~]$ mkdir tmp0
[ec2-user@ip-172-31-0-70 ~]$ cd tmp0/
[ec2-user@ip-172-31-0-70 tmp0]$ touch test_{1..20}.log
[ec2-user@ip-172-31-0-70 tmp0]$ mv ../myscript.sh .
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 4
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_10.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_11.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_12.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_13.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_14.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_15.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_16.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_17.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_18.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_19.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_1.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_20.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_2.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_3.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_4.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_5.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_6.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_7.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_8.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_9.log
[ec2-user@ip-172-31-0-70 tmp0]$ ./myscript.sh 
19 - 1 = 18
19 - 2 = 17
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 12
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
-rw-rw-r-- 1 ec2-user ec2-user  32 Aug  9 12:24 test_10.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  32 Aug  9 12:24 test_11.log.gz
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_12.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_13.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_14.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_15.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_16.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_17.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_18.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_19.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_1.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_20.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_2.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_3.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_4.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_5.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_6.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_7.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_8.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:24 test_9.log
[ec2-user@ip-172-31-0-70 tmp0]$ 
[ec2-user@ip-172-31-0-70 tmp0]$ 
[ec2-user@ip-172-31-0-70 tmp0]$ bash -x ./myscript.sh 
++ wc -l
++ ls -lah ./test_12.log ./test_13.log ./test_14.log ./test_15.log ./test_16.log ./test_17.log ./test_18.log ./test_19.log ./test_1.log ./test_20.log ./test_2.log ./test_3.log ./test_4.log ./test_5.log ./test_6.log ./test_7.log ./test_8.log ./test_9.log
+ TOTAL=18
+ let TOTAL--
+ COUNT=0
+ [[ 0 < 17 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_12.log
+ let COUNT++
+ echo '17 - 1 = 16'
17 - 1 = 16
+ sleep 2
+ [[ 1 < 17 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_13.log
+ let COUNT++
+ echo '17 - 2 = 15'
17 - 2 = 15
+ sleep 2
+ [[ 2 < 17 ]]
[ec2-user@ip-172-31-0-70 tmp0]$

You can see that it stops while 2 is less than 17. But, if when setting up the tmp0 directory, you only touch 1 through 9 instead of double digits, the while loop runs all the way through:

[ec2-user@ip-172-31-0-70 tmp0]$ rm test*
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 4
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
[ec2-user@ip-172-31-0-70 tmp0]$ touch test_{1..9}.log
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 4
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_1.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_2.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_3.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_4.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_5.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_6.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_7.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_8.log
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_9.log
[ec2-user@ip-172-31-0-70 tmp0]$ bash -x ./myscript.sh 
++ wc -l
++ ls -lah ./test_1.log ./test_2.log ./test_3.log ./test_4.log ./test_5.log ./test_6.log ./test_7.log ./test_8.log ./test_9.log
+ TOTAL=9
+ let TOTAL--
+ COUNT=0
+ [[ 0 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_1.log
+ let COUNT++
+ echo '8 - 1 = 7'
8 - 1 = 7
+ sleep 2
+ [[ 1 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_2.log
+ let COUNT++
+ echo '8 - 2 = 6'
8 - 2 = 6
+ sleep 2
+ [[ 2 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_3.log
+ let COUNT++
+ echo '8 - 3 = 5'
8 - 3 = 5
+ sleep 2
+ [[ 3 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_4.log
+ let COUNT++
+ echo '8 - 4 = 4'
8 - 4 = 4
+ sleep 2
+ [[ 4 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_5.log
+ let COUNT++
+ echo '8 - 5 = 3'
8 - 5 = 3
+ sleep 2
+ [[ 5 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_6.log
+ let COUNT++
+ echo '8 - 6 = 2'
8 - 6 = 2
+ sleep 2
+ [[ 6 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_7.log
+ let COUNT++
+ echo '8 - 7 = 1'
8 - 7 = 1
+ sleep 2
+ [[ 7 < 8 ]]
++ head -n 1
++ grep .log
++ grep -v .gz
++ ls -1
+ gzip test_8.log
+ let COUNT++
+ echo '8 - 8 = 0'
8 - 8 = 0
+ sleep 2
+ [[ 8 < 8 ]]
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 36
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_1.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_2.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_3.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_4.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_5.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_6.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_7.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_8.log.gz
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_9.log
[ec2-user@ip-172-31-0-70 tmp0]$ bash -x ./myscript.sh 
++ wc -l
++ ls -lah ./test_9.log
+ TOTAL=1
+ let TOTAL--
+ COUNT=0
+ [[ 0 < 0 ]]
[ec2-user@ip-172-31-0-70 tmp0]$ ll
total 36
-rwxrwxr-x 1 ec2-user ec2-user 245 Aug  9 11:45 myscript.sh
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_1.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_2.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_3.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_4.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_5.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_6.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_7.log.gz
-rw-rw-r-- 1 ec2-user ec2-user  31 Aug  9 12:28 test_8.log.gz
-rw-rw-r-- 1 ec2-user ec2-user   0 Aug  9 12:28 test_9.log
[ec2-user@ip-172-31-0-70 tmp0]$

I'm just wondering why. I'm sure it's probably something stupid simple, but I'm just not seeing it. At first I thought it was because bash on macOS was "special", but then I tried it on Amazon Linux, and experienced the same thing.

Thanks in advance.

`[[ $count < $total ]]` is comparing ASCII value of strings, so `10` is less than `2`. Use `(( count < total ))`, `[[ $count -lt $total ]]`, or `[ "$count" -lt "$total" ]`. — Charles Duffy, Aug 09 '18 at 19:40
Thank you @CharlesDuffy. I apologize for not doing a better job searching. I did search, but apparently not for the correct terms. — amoreno, Aug 09 '18 at 20:22

score 2 · Accepted Answer · answered Aug 09 '18 at 19:40

2

You are doing string comparisons.

[[ '2' > '19' ]] && echo "true"

will output

true

Use integer arithmetic.

(( 2 > 19 )) && echo "true"

will not output anything.

answered Aug 09 '18 at 19:40

jeremysprofile

10,028
4
33
53

In [How to Answer](https://stackoverflow.com/help/how-to-answer), see the section "Answer Well-Asked Questions", particularly the bullet point regarding questions which "*have already been asked and answered many times before*". – Charles Duffy Aug 09 '18 at 19:42
1

I agree, I'm sure it is. I don't know of which question it is a duplicate, though, and I'm not sure what to search to find it. I did a best effort google search and the first 3 results were not ones I'd like to as a duplicate. At that point, this took 30s to write and was faster to answer the question than finding the right question to link to – jeremysprofile Aug 09 '18 at 19:43
I understand, but pointing someone to the most complete answer is better for long-term quality than being fast/first. (That said, I've sometimes made a habit of adding a community-wiki answer when I know something is duplicate but can rattle off a quick solution, and closing as dupe a few minutes later after spending the time to find a good list; community-wiki means others are allowed and encouraged to edit that answer to tune it for the individual OP, whereas the list of canonical answers points others with similar questions towards a more complete and canonical source). – Charles Duffy Aug 09 '18 at 19:50
@CharlesDuffy, so in the future, is it better to "give up" on a question that I can't find a link to, leave a comment that basically answers the question, or provide an answer? If I was a new user, linking to a question that I reasonably would not have known how to search for and marking mine as a duplicate would be irritating, especially as (I believe) it counts as "bad question" in SO, and it threatens to suspend you if you have a record of it. This happened to me on a few questions when I first joined. – jeremysprofile Aug 09 '18 at 19:54
@CharlesDuffy, I will keep the community wiki option in mind so I don't appear to be trawling for points. – jeremysprofile Aug 09 '18 at 19:55
Duplicates aren't innately bad -- there's a reason they can be upvoted: A good duplicate provides an extra set of search terms that direct to the existing version of the answer, which otherwise someone wouldn't know how to find. If you think a question adds a set of search terms that improves the knowledgebase by bringing a wider audience to an existing answer, by all means upvote it as well as flagging it dupe. :) – Charles Duffy Aug 09 '18 at 19:58
Thank you @jeremysprofile. As mentioned in my response to CharlesDuffy, I did search (just like you), and didn't immediately find what I thought I was looking for (just like you). So I appreciate your help, and I will do a better job in the future of searching. Clearly I didn't realize there was a difference between string and integer comparisons, so that wouldn't have been the first place I looked, for what that's worth. – amoreno Aug 09 '18 at 20:25
@amoreno, no problem. You did everything correctly, especially the amount of logging info you provided (I wouldn't have been able to answer as fast without seeing the term it failed on was 2 vs 19 b/c I forget about string vs integer comparisons too). – jeremysprofile Aug 09 '18 at 20:29

while loop stops before false

1 Answers1