Bash find replace multiple files to NEW files

Question

I have a directory of let's call them template files. They have one value in them that needs to be substituted. I wanted a pristine folder with a 'keyword' in the substitution place in each file. I wanted to substitute the word and output all files to a new folder with files named the same as the input. The command that I found here is:

find . -name '*.sql' -i -exec sed -e 's/TEXTTOSUBSTITUTE/SUBSTITUTIONTEXT/g' {} \;

How can I get this command to output files to another directory and leave the originals unchanged? thank you so much for your help! I want the original files to remain unchanged. the folder is called Templates and the folder in the same directory as the directory Templates is called Usefiles. The template files act as the base files and so should not change. I guess I could use the suggestion above, but do 2 separate find commands, one to copy and the other to sed in the new directory UPDATE>>>> I've been trying the suggestions, but the sed doesn't seem to work. The copy works, but the sed is a variable and is coming out blank in the script. Here is my current, and thanks again!! :

#!/bin/bash

echo "Enter Year i.e. 2020"
read -p reportyear
echo "Enter Month i.e. 04"
read -p reportmonth
SUBSTITUTIONTEXT=$reportyear/$reportmonth

find MOT/ -name '*.sql' -exec sh -c 'for f; do
 echo cp -v "$f" MOU/ && echo sed -ie "s/TEXTTOSUBSTITUTE/$SUBSTITUTIONTEXT/g" MOU/"${f##*/}"
done' _ {} +

FINAL UPDATE Thanks to you guys, a working solution:

echo "Enter Year i.e. 2020"
read reportyear
echo "Enter Month i.e. 04"
read reportmonth
SUBSTITUTIONTEXT=$reportyear\/$reportmonth
cd /to/relevant/files
find MOT/ -name '*.sql' -exec sh -c 'for f; do 
cp -v "$f" MOU/ && sed -i 's@SUBSTITUTIONYEARMONTH@"$SUBSTITUTIONTEXT"@' MOU/"${f##*/}" 
done' _ {} + > /dev/null 2>&1

Is *another directory* in the same folder as .sql files? And is `find` really required here, are there .sql files in subdirectories? — oguz ismail, Mar 27 '20 at 07:57
My answer currently the original files is copied to destination unchanged and edit the files in the current directory. That is how I understood your question. — Jetchisel, Mar 27 '20 at 08:05
Use a different delimiter for sed e.g. `"s,TEXTTOSUBSTITUTE,$SUBSTITUTIONTEXT,g"` basically the delimiter must not be in the string/regularexpression so you can choose any string, like a pipe `|` or what I have shown you a comma `,`. — Jetchisel, Mar 27 '20 at 09:31

Jetchisel · Accepted Answer · 2020-03-27T09:13:02.383

1

As described by the the question two directories Template as the source and Usedfiles as the destination.

Using find(1)

find Templates/ -name '*.sql' -exec sh -c 'for f; do
 echo cp -v "$f" Usefiles/ && echo sed -ie "s/TEXTTOSUBSTITUTE/SUBSTITUTIONTEXT/g" Usefiles/"${f##*/}"
done' _ {} +

Another option is to use the shellopt globstar from bash

#!/usr/bin/env bash

shopt -s globstar

for f in ./Templates/**/*.sql; do
 echo cp -v "$f" Usefiles/ && echo sed -ie 's/TEXTTOSUBSTITUTE/SUBSTITUTIONTEXT/g' Usefiles/"${f##*/}"
done

Remove all the echo If you're satisfied with the result so It can actually do what needs to be done.
See Loop through files using find and Using Find
See Parameter Expansion for the ${f##*/}
See Globstar for the shopt -s globstar

edited Mar 27 '20 at 09:13

answered Mar 27 '20 at 07:47

Jetchisel

7,493
2
19
18

Right, need to clarify from the OP, – Jetchisel Mar 27 '20 at 07:58
I want the original files to remain unchanged. the folder is called Templates and the folder in the same directory as the directory Templates is called Usefiles. The template files act as the base files and so should not change. I guess I could use the suggestion above, but do 2 separate find commands, one to copy and the other to sed in the new directory. – mAsTeRoFnOne Mar 27 '20 at 08:28
Could you please update the question and add what you have posted here. – Jetchisel Mar 27 '20 at 08:42

score 1 · Answer 2 · answered Mar 27 '20 at 14:16

This might work for you (GNU sed and parallel):

sub=2020/04
find Templates -name '*.sql'| parallel "sed 's#key#'"$sub"'#g' {} > Usefiles/{/}"

Set the substitute parameter in sub.

Pipe the find command to the parallel invocation.

Run sed against each file from the find command and substitute $sub for each key placing the output in a the directory Usefiles in the same directory as Templates with the same file name.

Bash find replace multiple files to NEW files

2 Answers2