Bash for loop stops after one iteration without error

Question

Lets say I have a file dates.json:

2015-11-01T12:01:52
2015-11-03T03:58:57
2015-11-09T02:43:59
2015-11-10T08:22:00
2015-11-11T05:14:51
2015-11-11T12:47:02
2015-11-13T08:33:40

I want to separate the rows to different files according to the date. I made the following script:

#!/bin/bash
set -e
file="$1"
for i in $(seq 1 1 31); do
    if [ $i -lt 10 ]; then
      echo 'looking for 2015-11-0'$i
      cat $file | grep "2015-11-0"$i > $i.json
    else
      echo 'looking for 2015-11-'$i
      cat $file | grep "2015-11-"$i > $i.json
    fi
done

When I execute I get the following:

$ bash example.sh dates.json
looking for 2015-11-01
looking for 2015-11-02

If I try without the cat... rows the script prints all the echo commands, and if I try only the cat | grep command on the command line it works.

Would you know why does it behave like this?

Comment out `set -e` and use `bash -x example.sh dates.json` — anubhava, Dec 15 '15 at 18:44
Why do you use a directive (`set -e`) if you don't know what it does? — Charles Duffy, Dec 15 '15 at 18:56
@CharlesDuffy I just wanted to stop the script in any error case, do you mean I don't know what `set -e` does or something else? — dtj, Dec 16 '15 at 18:04
Yes, `set -e` stops on any error, and `grep` finding no matches in a file is an error. So it's doing exactly what you tell it to. — Charles Duffy, Dec 16 '15 at 18:06
BTW, your expansions of `$i` should be *inside* quotes. Which is to say: `grep "2015-11-0$i" "$file" >"$i.json"`; otherwise, those expansions are prone to string-splitting and globbing. — Charles Duffy, Dec 16 '15 at 18:08
See also [useless use of `cat`.](https://stackoverflow.com/questions/11710552/useless-use-of-cat) — tripleee, Aug 01 '20 at 15:02

score 4 · Accepted Answer · answered Dec 15 '15 at 18:59

4

If you need set -e in other parts of the script, you need to handle grep not to stop your script:

# cat $file | grep "2015-11-0"$i > $i.json
grep "2015-11-0"$i "$file" > $i.json || :

answered Dec 15 '15 at 18:59

choroba

231,213
25
204
289

Certainly answers the question correctly -- but might be worth fixing the quoting while you're at it. – Charles Duffy Dec 16 '15 at 18:08
@CharlesDuffy What is the problem with the quoting? . @choroba What does `|| :` do in this case? – dtj Dec 16 '15 at 18:13
1

@dtj, `:` is a synonym for `true`. The problem with the quoting is that unquoted expansions are string-split and glob-expanded, and you probably don't want to string-split or glob-expand `$i`. – Charles Duffy Dec 16 '15 at 18:13

score 3 · Answer 2 · answered Dec 15 '15 at 18:57

3

set -e forces script to exit if command exits with non-zero status.

+

grep returns 1 if it fails to find match in file.

+

dates.json has no 2015-11-02

=

error

answered Dec 15 '15 at 18:57

Sergei

1,599
12
21

Thanks! I thought the grep failing to find a match would not count as an error to the `set -e` option – dtj Dec 16 '15 at 18:11
@dtj, if failure to match wasn't reported in grep's exit status, then you couldn't use `if grep -q -e pattern file; then ...` to tell if a match existed. – Charles Duffy Dec 16 '15 at 18:15
...and, well, UNIX exit status doesn't make a distinction between "successful-true", "successful-false" and "failed"; it's just zero (success) or nonzero (failure) -- any more fine-grained definition is application-specific. – Charles Duffy Dec 16 '15 at 18:16

score 0 · Answer 3 · answered Dec 15 '15 at 18:49

0

it's because of our set -e which causes the script to exit. Remove this line, then it should work

answered Dec 15 '15 at 18:49

Raphael Roth

26,751
15
88
145

Bash for loop stops after one iteration without error

3 Answers3