Bash script - perform grep on filenames containing particular numbers

Question

I have a bunch of files such as:

jobReports/NARX_20191212_1.out
jobReports/NARX_20191212_10.out
jobReports/NARX_20197695_2.out
jobReports/NARX_20197695_3.out
jobReports/NARX_20261798_1.out

where the first numeral sequence denotes a jobID, and the second numeral sequence denotes the arrayID.

I want to perform a search on all the .out files in that directory with jobID equal to 20197695 or 20261798.

In the command line, this works:

grep "Saving results to" jobReports/NARX_{20197695,20261798}*.out | wc -l

, which gives me a count of occurences of that string in the desired files. In my script I have the following:

#!/bin/bash
myJobs = ( 20197695, 20261798 )
IFS=, eval 'myJobs_string="${myJobs[*]}"'
echo $(grep "Saving results to" jobReports/NARX_{$myJobs_string}*.out |wc -l)

when running, I get the error:

grep: jobReports/NARX_{: No such file or directory
grep: 20197695: No such file or directory
grep: }*.out: No such file or directory

So looks like inside the script, the brace expansion is not working as I expected...what can I do to fix this?

why myJobs is an array and not a simple variable "{20197695, 20261798}" — kyodev, Apr 08 '18 at 04:10
You cannot use a variable inside braces for brace expansion. When you do, the `','` loses all significance and is simply part of a string. Just loop over your array and keep a `sum` of the lines resulting your `grep ... | wc -l` for each array member. — David C. Rankin, Apr 08 '18 at 10:46
@kyodev Good point, I did not think of that. But it (the solution you posted) still gives me the same problem — Asy, Apr 09 '18 at 22:53
@DavidC.Rankin So is there no way to do brace expansion with an array for grep as I wanted to? — Asy, Apr 09 '18 at 22:54
double quote around variable: `jobReports/NARX_{"$myJobs_string"}*.out` — kyodev, Apr 10 '18 at 00:14
`"$myJobs_string"` is only `"${myJobs_string[@]}"` and with your `myJobs_string="${myJobs[*]}`, myJobs_string is not an arrey — kyodev, Apr 10 '18 at 02:38
Not using an array at all would seem like the obvious, trivial workaround in this particular case. `grep "Saving results to" jobReports/NARX_20197695*.out jobReports/NARX_20261798*.out` — tripleee, Apr 10 '18 at 03:31

kyodev · Answer 1 · 2018-04-10T02:00:54.693

-1

#!/bin/bash

myJobs=( 20197695, 20261798 )
ifs_ini="$IFS"
IFS+=','
myJobs_string=( ${myJobs[*]} )
IFS="$ifs_ini"

echo myJobs ${myJobs[@]}
echo myJobs_string ${myJobs_string[@]}
echo jobReports/NARX_{${myJobs_string[@]}}*.out
grep -c 'Saving results to.*$' jobReports/NARX_{"$myJobs_string"}*.out
bash --version

output

myJobs 20197695 20261798
myJobs_string 20197695 20261798
jobReports/NARX_{20197695 20261798}*.out
<not tested>

PS: grep -c ... is the same as echo $( echo grep ... | wc -l )

edited Apr 10 '18 at 02:00

answered Apr 08 '18 at 04:30

kyodev

573
2
14

I tried the following in my script: `myJobs=( 20367893 ); ifi_ini="$IFS"; IFS+=','; myJobs_string=( ${myJobs[*]} ); IFS="$ifs_ini"; echo 'Total ' $(grep "Saving results to" jobReports/NARX_{"$myJobs_string"}*.out | wc -l) ' completed jobs';` The error I get is: `grep: jobReports/NARX_{20367893}*.out: No such file or directory` As a check: `grep "Saving results to" jobReports/NARX_{20367893}*.out |wc -l` gives me a correct count. – Asy Apr 10 '18 at 00:46
please, use for test my code. In yours, it is not a array for final display. Use indentation, no code here. Check with https://www.shellcheck.net/. @Asy – kyodev Apr 10 '18 at 01:58

Bash script - perform grep on filenames containing particular numbers

1 Answers1