1

A stupid question to ask however I don't seem to be able to find the answer through stackoverflow or other googled results. Either this question is too specific or too dumb. I think it's the later one...

I am using awk to filter files that are under different subdirectory and I want to output the results under the same subdirectory where the file is located.

I was playing with the find command and was able to awk the files in all the subdirectory that matches the pattern.

I find files ended with .tsv and then awk the 8th column (values that under -1) and then I output the file use the base of the file name and add .txt

for file in $(find $DIR2 -type f -name '*.tsv'); do
awk -F$'\t' '$8 < -1' $(find $DIR2 -type f -name '*.tsv') > $DIR2/$(basename ${file%.*}).txt; done

However, I am kind of stuck, not knowing how to output the files under the original subdirectory.

If I use $DIR2/ it only export under this specific directory and without the $DIR2/, it gets exported to the current working directory.

Molly_K
  • 313
  • 2
  • 13
  • This is very helpful for understanding the exec {}+ command https://unix.stackexchange.com/questions/389705/understanding-the-exec-option-of-find – Molly_K Mar 22 '19 at 21:12

1 Answers1

1

Try this (untested):

find "$DIR2" -type f -name '*.tsv' -exec \
awk -F'\t' 'FNR==1{close(out); out=FILENAME; sub(/[^.]+$/,"txt",out)} $8 < -1{print > out}' {} +

If your find doesn't support + then replace it with \;.

Ed Morton
  • 188,023
  • 17
  • 78
  • 185
  • 1
    Thank you - I corrected it to directory. I am going to test your code now! – Molly_K Mar 18 '19 at 15:47
  • It works well in my test directory! Thank you. Could you expand and explain a little what the code mean so that I could learn from it and use it in the future? Thank you. especially the "close(out)" and "{} +" part. – Molly_K Mar 18 '19 at 15:50
  • And I really like your suggestion as it does not involve a for loop. – Molly_K Mar 18 '19 at 15:52
  • 1
    You're welcome. I really think it's pretty obvious what the code does though. Did you try looking up `close()` in the awk man page and `{} +` in the find one? `close(out)` is closing the file we just finished writing to and `{} +` represents the list of files that `find` is passing to `awk` in batches. – Ed Morton Mar 18 '19 at 16:03
  • 1
    Thank you for pointing me to the close() and awk syntax for enclosing {} - I am looking at them now. Thanks again! – Molly_K Mar 18 '19 at 16:08
  • This post is quite helpful. https://stackoverflow.com/questions/6085156/using-semicolon-vs-plus-with-exec-in-find – Molly_K Mar 18 '19 at 18:24