-1

I have a list of files in ids1. Example 

s1_aaa
s1_aab
s1_aac

I want to submit jobs for these files using sbatch: I use

for x in `cat ids1`;do sbatch batch_job.sbatch $x ;done

This works fine.

Now I try for 2 variables in the for loop. Example I have another file ids2 like 

s2_aaa
s2_aab
s2_aac

I want to submit jobs for files in ids1 and ids2 simultaneously i.e. pairs (s1_aaa,s2_aaa), (s1_aab,s2_aab), (s1_aac,s2_aac) and so on. When I try 

for x,y in `cat ids1;cat ids2`;do sbatch batchjob.sbatch $x $y ;done

I get the error 

-bash: `x,y': not a valid identifier

What am I doing wrong?

Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
Ssank
  • 3,367
  • 7
  • 28
  • 34

2 Answers2

2

Read each into an array:

# in bash 4.0 or newer
readarray -t ids1 <ids1 # read each line from file ids1 into array ids1
readarray -t ids2 <ids2 # read each line from file ids2 into array ids2

...or, in bash 3.x:

ids1=( ); ids2=( )
while IFS= read -r id; do ids1+=( "$id" ); done <ids1
while IFS= read -r id; do ids2+=( "$id" ); done <ids2

...and iterate over indexes:

for idx in "${!ids1[@]}"; do # iterate over indexes in ids1
  id1=${ids1[$idx]} # and use that index to look up items in *both* arrays
  id2=${ids2[$idx]}
  echo "Processing $id1 and $id2"
done

Another option is to use paste to combine both files into a single stream, and use a BashFAQ #1 while read loop to iterate over the two combined columns:

while IFS=$'\t' read -r id1 id2; do
  echo "Processing $id1 and $id2"
done < <(paste ids1 ids2)
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
0

if the lines match in the two files, you can paste them first and iterate on them pairwise, i.e., 

while read a b; do sbatch batchjob.sbatch $a $b; done < <(paste ids1 ids2)
karakfa
  • 66,216
  • 7
  • 41
  • 56
  • Believe I beat you there (albeit in an edit that got folded into my other edits, so it's hard to see the timeline). – Charles Duffy Aug 08 '16 at 18:50
  • perhaps, yours is more comprehensive and longer to write, even if not commit time you beat in answer time for sure. – karakfa Aug 08 '16 at 18:51