2

I have a directory named ../words that contains a number of subdirectories(words/blabla,words/blabla), in turn each subdirectory contains two types of files (.*1.txt) and (.*2.txt) . What I need, it is to execute an AWK script against each one of these files.

Could it be something like?

for d in words/*
do
    for f in .*[0-9].txt
    do
    awk -f script.awk ${f}
    done
done
Firefly
  • 449
  • 5
  • 20
  • 1
    So have you tried that? Problem is that you are not using `$d`. You could also use the `find` command. – cdarke Feb 26 '16 at 16:28

3 Answers3

2

If you want to keep your for statement structure and apply the awk script to each specified file, you can do the following:

for file in $(find words -type f -name ".*[12].txt"); do
    awk -f script.awk "$file"
done

The find command is useful for recursively looking through a directory for any pattern of files.

Edit: If your file names contain things like spaces, the above script may not process them properly, so you can do the following instead:

find words -type f -name ".*[12].txt" -print0 | while read -d $'\0' file
do 
    awk -f script.awk "$file"
done

or using xargs:

find words -type f -name ".*[12].txt" -print0 | xargs -0 awk -f script.awk

This allows you to delimit your file names with null \0 characters, so variations in name spacing or other special characters will not be a problem. (You can find more information here: Filenames with spaces breaking for loop, and find command, or here: Handling filenames with spaces, or here: loop through filenames returned by find).

assefamaru
  • 2,751
  • 2
  • 10
  • 14
  • Never do do `for file in $(whatever)` as it fails for various file names such as those that contain spaces. Always do `whatever | while IFS= read -r file` instead or even better `whatever | xargs`. – Ed Morton Feb 26 '16 at 16:54
  • 1
    You are right! And I made an edit above to offer an alternative as you suggested. – assefamaru Feb 26 '16 at 17:23
  • @EdMorton: Those suggestions still break for some filenames. (Note that filenames can even contain newlines.) – ruakh Feb 26 '16 at 17:26
  • @ruakh The loop syntax I suggested will ONLY break for file names containing newlines and the xargs one doesn't even break for that when used with a `whatever` than can output nul-separated instead of newline-separated strings, e.g. `find -print0 | xargs -0`, hence "**even better** `whatever | xargs`". – Ed Morton Feb 26 '16 at 17:31
1

Given what you've told us so far, this should be all you need:

awk -f script.awk ../words/blabla/.*[12].txt
Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • I dunno. `script.awk` might sincerely behave differently if all files are passed in a single call, vs. if each file is passed in a separate call. – ruakh Feb 26 '16 at 16:32
  • Definitely and there are other considerations too but the OP has given us no clue what `script.awk` does, nor provided any sample input or expected output or even a concrete representation of her directory structure - hence "Given what you've told us so far" – Ed Morton Feb 26 '16 at 16:34
1

If you need to skip the intermediate directory level and just look under the subdirectories you can use max/min depth

$ find words -maxdepth 2 -mindepth 2 -type f -name '*[0-9].txt' | xargs awk -f ...
karakfa
  • 66,216
  • 7
  • 41
  • 56