Input folder / output folder for each file in AWK

Question

I am trying to run (several) Awk scripts through a list of files and would like to get each file as an output in a different folder. I tried already several ways but can not find the solution. The output in the output folder is always a single file called {} which includes all content of all files from the input folder.

Here is my code:

input_folder="/path/to/input"
output_folder="/path/to/output"

find $input_folder -type f -exec awk '! /rrsig/ && ! /dskey/ {print $1,";",$5}' {} >> $output_folder/{} \;

Can you please give me a hint what I am doing wrong? The code is called in a .sh script.

Naming a file `{}` or any other symbols isn't as useful for future maintenance as naming it something meaningful that tells you what it contains and is more likely to cause future scripts to break if they're not written carefully. At best it'll lead to confusing code that's using `{}` as a file name where we're all used to seeing `{}` be a placeholder for a file name in `find`, `xargs`, etc. — Ed Morton, Dec 28 '22 at 14:03
I tried to explain in my question, that I need the original file names in my output folder. But my script is generating a folder with the name {}. That is exactly the issue I have :-) — NightShift, Dec 28 '22 at 14:47
You said `The output in the output folder is always a single file called {} which includes all content of all files from the input folder` - I thought you were telling us your requirement, not telling us about a problem. Please [edit] your question to clearly state your requirements, the problem you're having with your existing script, and provide concise, testable sample input and expected output (e.g. a couple of small input files and the associated output files). — Ed Morton, Dec 28 '22 at 14:51
The redirection happens when the shell parses your command line, before `find` runs. To perform a redirection within `-exec` you have to wrap it somehow. The proposed duplicate has a couple of solutions to that. — tripleee, Dec 28 '22 at 18:43

markp-fuso · Answer 1 · 2022-12-28T17:18:15.553

I'd probably opt for a (slightly) less complicated find | xargs, eg:

find "${input_folder}" -type f | xargs -r \
awk -v outd="${output_folder}" '
FNR==1                 { close(outd "/" outf); outf=FILENAME; sub(/.*\//,"",outf) }
! /rrsig/ && ! /dskey/ { print $1,";",$5 > (outd "/" outf) }'

NOTE: the commas in $1,";",$5 will insert spaces between $1, ; and $2; if the spaces are not desired then use $1 ";" $5 (ie, remove the commas)

Input folder / output folder for each file in AWK

1 Answers1