0

I have a script, /home/user/me/my_script.sh that is supposed to loop over multiple directories and process files. My current working directory is /home/user/me. A call to ls -R yields:

./projects:

dir1 dir2 dir3

./projects/dir1:

image1.ntf points2.csv image1.img image1.hdr

./projects/dir2:

image2.ntf points2.csv image2.img image2.hdr

./projects/dir3:

image3.ntf points3.csv image3.img image3.hdr

I have this script:

#! /bin/bash -f
for $dir in $1*
do
  echo $dir
  set cmd = `/home/tools/tool.sh -i $dir/*.ntf -flag1 -flag2 -flag3 opt3`
  $cmd
done

This is how it is run (from cwd /home/user/me) and the result:

bash-4.1$ ./myscript.sh projects/
projects/*
bash-4.1$

This is not the expected output.The expected output is:

bash-4.1$ ./myscript.sh projects/
projects/dir1
[output from tool.sh]
projects/dir2
[output from tool.sh]
projects/dir3
[output from tool.sh]
bash-4.1$

What should happen is the script should go into the first directory, find the *.ntf file and pass it to tool.sh. At that point I would start seeing output from that tool. I have run the tool on a single file:

bash-4.1$ /home/tools/tool.sh -i /home/user/me/projects/dir1/image1.ntf -flag1 -flag2 -flag3 opt3
[expected output from tool. lengthy.]
bash-4.1$

I have tried syntax found here: How to loop over directories in Linux? and here: Looping over directories in Bash

for $dir in /$1*/
do ...

Result:

bash-4.1$ ./myscript.sh projects/
/projects/*/

And:

for $dir in $1/*
do ...

Result:

bash-4.1$ ./myscript.sh projects
projects/*

I'm not sure how many other iterations of wildcard and slash I can come up with. What is the correct syntax?

SatelliteEyes
  • 75
  • 1
  • 8
  • Is the lengthy expected output from `tool.sh` a set of commands, or some other output? – ghoti Jun 28 '18 at 19:40
  • It is text output about where the tool is in the process. It takes a *.ntf file and performs several calculations on the image to produce a calibrated image. A large image could take several minutes to process, so this output keeps the user informed about the progress. – SatelliteEyes Jun 28 '18 at 20:03
  • 1
    So .. you appear to be trying to set the variable `$cmd` to the output of the script, and then running that output as if it were a command. What is your intent with that variable? Why not just run the tool directly? (Feel free to answer by clarifying in your question rather than comments.) – ghoti Jun 28 '18 at 20:26
  • BTW, http://shellcheck.net/ would catch at least a subset of the issues automatically. – Charles Duffy Jun 28 '18 at 20:30

1 Answers1

2

First, you should remove flag -f in your shebang, because it utterly means:

$ man bash
[…]
        -f      Disable pathname expansion.

Second, there are some typical bug patterns: spaces missing around variables (write "$dir" to cope with directory names containing spaces), there is a spurious $ in your for line (write for dir in "$1"*) instead, the set line is incorrect (set is a shell builtin only used to change the configuration of the shell, e.g., set -x), according to your answer to @ghoti's question it seems that the $cmd line is unnecessary. Also, the backquotes syntax is deprecated and could have been replaced with cmd=$(/home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3).

This would lead to the following script:

#!/bin/bash
for dir in "$1"*
do
  [[ -d "$dir" ]] || continue  # only consider existing folders
  printf "%s=%q\n" dir "$dir"
  /home/tools/tool.sh -i "$dir"/*.ntf -flag1 -flag2 -flag3 opt3
done

As an aside, I would recommend to always run the ShellCheck static analyzer on your Bash scripts, in order to detect typical bugs and have feedback w.r.t. good practices. If you have a Linux distribution, it should be installable with your standard package manager.

ErikMD
  • 13,377
  • 3
  • 35
  • 71
  • 1
    `"$1"*/`, if you want to only have directories as glob results. Otherwise files can show up too. Could alternatively use `[[ -d "$dir" ]] || continue` inside the loop -- that would also fix the case where no directories exist so you get a string with a literal `*` returned, without needing `nullglob`. – Charles Duffy Jun 28 '18 at 20:31
  • Indeed, @CharlesDuffy (thanks!) I've just added your second suggestion in my answer. – ErikMD Jun 28 '18 at 20:40
  • ...oh, might want to put a `\n` on the end of the `printf` format string. (Looks like something I may have written -- apologies if that thinko came from me!) – Charles Duffy Jun 28 '18 at 20:41
  • Thank you, this works! I will use ShellCheck extensively going forward. I am often building new scripts from other peoples scripts, so old practices get propagated forward when they should not. – SatelliteEyes Jun 28 '18 at 20:54