Run command for each line in file while file content is parameter

Question

I have a file that contains path names to directories like this:

~/subdirectory1/subdirectory2/.../a/
~/subdirectory1/subdirectory2/.../b/
...
~/subdirectory1/subdirectory2/.../z/

Wanted steps:

• Open textfile
• Fetch a line
• Pass it to command "ls"
• Pipe output and exclude all occurrences of ending "jpg" or "JPG"
• Route result into a new textfile (e.g. "_command_output.log")

I was playing around with xargs, but could not make it run.

Here is what I have so far: List files and exclude all "jpg" or "JPG" results.

ls ~/subdirectory1/subdirectory2/.../a/ | awk '!/(jpg|JPG)/'

I put that into a loop:

for i in $(cat ~/textfile-with-pathnames.txt) do
   #do your stuff to $i here
   ##xargs ls;
   ls ${i} | awk '!/(jpg|JPG)/' 
done

Another try:

cat ~/textfile-with-pathnames.txt  | while read line; 
do 
   ls "$line" | awk '!(jpg|JPG)'; 
done

How can I run an argument for each line in the file and use the file content as parameter/argument?

Answer 1

Here is one solution (to my knowledge, this is fully POSIX compliant):

#!/usr/bin/env sh

while read -r line; do
    # Expand ~ in line (see below if you want details on how this works)
    line="$line/"
    line="${HOME}/${line#"~/"}"

    # ls line, then inverse case-insensitive grep, and append to file
    ls "$line" | grep -iv '[.]jpg$' >> _command_output.log
done < directories.txt

If you want to include hidden files/directories in output (i.e., if you want to include files/directories with filenames starting with .), replace ls with ls -a.

The -i option of grep turns on case-insensitivity, and -v turns on inverse matching (both -i and -v are POSIX options).

My script assumes that directories.txt only contains directories.

---

Tangential: What the heck does line="${HOME}/${line#"~/"}" do?

The complicated line="${HOME}/${line#"~/"}" is based on this answer.

Here is an explanation, in case my link dies one day. When the shell interpreter looks at line="${HOME}/${line#"~/"}", it sees:

1. ${HOME} and ${line#"~/"} must be expanded.
2. ${HOME} is expanded to the content of the $HOME variable.
3. ${line#"~/"} triggers a nested expansion: "~/" is parsed but, being quoted, is treated as a literal. You could have used single quotes here with the same result.
4. ${line#"~/"} expression itself is now expanded, resulting in the prefix ~/ being removed from the value of $line.
5. The result of the above is now concatenated: the expansion of ${HOME}, the literal /, the expansion ${line#"~/"}.
6. The end-result is enclosed in double-quotes, functionally preventing word splitting.