bash for loop: failed to open file

Question

I have many directories with multiple files, each following strict naming convention. I tried this loop:

for s in /home/admin1/phd/results_Sample_*_hg38_hg19/Sample_*_bwa_LongSeed_sorted_hg38.bam; do 
    bedtools genomecov -ibam $s > $s
done

but get for every file:

Failed to open file

I just can't see the mistake

the problem has nothing to do with the loop - http://stackoverflow.com/help/mcve — Karoly Horvath, Jul 31 '15 at 08:46

hek2mgl · Accepted Answer · 2015-07-31T08:54:07.317

Note that if you are redirecting the output to the input file, bash will truncate the file before the process starts. Like this:

cat file > file

will result an empty file. Meaning you'll loose it's contents.

This is because bash will open all files which are subject of io redirection before it starts the command. If you are using > bash will additionally truncate the file to zero length. I don't know bedtools but I guess it expects non-empty files as input, that's why the message.

About the loop. Usually it is not a good idea to loop over the results of a glob expression like you did. This is because file names may contain the internal field separator of bash which would lead to inconsistent results.

I would suggest to use find instead:

find /home/admin1/phd/results_Sample_*_hg38_hg19 \
    -maxdepth 1 \
    -name 'Sample_*_bwa_LongSeed_sorted_hg38.bam' \
    -exec bash -c 'bedtools genomecov -ibam "{}" > "{}.copy"' \;

how can I specify output directory for .copy files? – Irek Jul 31 '15 at 12:22 — Irek, Jul 31 '15 at 12:22
Simply use `> "path/to/{}.copy"` – hek2mgl Jul 31 '15 at 13:17 — hek2mgl, Jul 31 '15 at 13:17

bash for loop: failed to open file

1 Answers1