How can I pass a file argument to my bash script twice and modify the filename

Question

We have a large number of files in a directory which need to be processed by a program called process, which takes two aguments infile and outfile.

We want to name the outfile after infile and add a suffix. E.g. for processing a single file, we would do: process somefile123 somefile123-processed

How can we process all files at once from the command line? (This is in a bash command line)

https://stackoverflow.com/questions/10523415/execute-command-on-all-files-in-a-directory — folibis, Sep 05 '21 at 16:58
@OliverGaida: It looks like a typo, but it's not. This takes into account the requirement "*process all files at once*". To process the files one after the other, you would have to replace `&` with `;`, as you suggested. — Cyrus, Sep 05 '21 at 20:22

score 1 · Answer 1 · answered Sep 11 '21 at 12:24

As @Cyrus says in the comments, the usual way to do that would be:

for f in *; do process "$f" "${f}-processed" & done

However, that may be undesirable and lead to a "thundering herd" of processes if you have thousands of files, so you might consider GNU Parallel which is:

more controllable,
can give you progress reports,
is easier to type

and by default runs one process per CPU core to keep them all busy. So, that would become:

parallel process {} {}-processed ::: *

How can I pass a file argument to my bash script twice and modify the filename

1 Answers1