Grouping files by directory branch/leaf

Question

I am new to bash and am having difficulty processing directory tree structures. I have a directory structure that is as follows

I/A/dataA.dat
I/B/dataB.dat
I/C/dataC.dat

II/A/dataA.dat
II/B/dataB.dat
II/C/dataC.dat

III/A/dataA.dat
III/B/dataB.dat
III/C/dataC.dat

I now want to process I/A/dataA.dat with I/B/dataB.dat and likewise II/A/dataA.dat with II/B/dataB.dat etc (for every case). But I do not want to process, for example, I/A/dataA.dat with II/B/dataB.dat.

What is the best way to find such pairs of files?

I wish to pass the names of the found pair to a bash function such as follows

function process_pair()
{
  fileOne=$1; #for example II/A/data.dat
  fileTwo=$2; #for example II/B/data.dat
  //the rest of the function that processes this pair.
}

but I do not know how to get the variables $1 and $2.

I am currently using bash and starting from the dirs=find $SOURCE -type d -links 2 command (to find all the leaf directories - adapted from this question). I am then trying to loop over these (using substring commands to obtain the directory above). However, I am finding this difficult and I think there must be a better way.

This is edging toward the sort of thing you might prefer a fuller scripting language for. The obvious method is to build up a data structure which, for each filename, contains a list of the directories it's appeared in, so that each time you read a directory, you process each file in it along the same-named file in each directory in the corresponding list. This would be really easy in perl, with a hash of arrays, for example. — Cascabel, Apr 06 '11 at 14:05
Thanks @Jefromi, I will dig out my Perl book if I get no more replies here (it needs a dusting anyway...) — Tom, Apr 06 '11 at 14:11

score 1 · Accepted Answer · answered Apr 06 '11 at 15:37

Readily done in the shell:

for dir in I II III; do
    subdirs=()
    for subdir in $dir/*; do subdirs+=("${subdir##*/}"); done
    for (( i=0 ;i<${#subdirs[*]}-1; i++ ));  do
        for (( j=i ;j<${#subdirs[*]} ;j++ ));  do
            a=${subdirs[$i]}
            b=${subdirs[$j]}
            process_pair $dir/$a/data$a.dat $dir/$b/data$b.dat
        done
    done
done

Grouping files by directory branch/leaf

1 Answers1