xargs: variable substitution after redirection

Question

I'm trying to find all text files which have the encoding iso-8859-1 and convert these to UTF-8. My attempt so far is:

find . -name '*.txt' | xargs grep 'iso-8859-1' | cut -d ':' -f1 | 
xargs iconv -f ISO-8859-1 -t UTF-8 {} > {}.converted

The (obvious) problem is that the last variable substitution won't work, since {} occurs after the redirection, and doesn't belong to xargs. As is I only get one file called {}.converted, not a.txt.converted, b.txt.converted etc. How can I make this work?

Note: I'm doing this on Cygwin, where iconv doesn't seem to support -o.

Please have a look at this [related question](http://stackoverflow.com/q/845863/183066). — jcollado, Jan 24 '12 at 12:20
I don't know which answer to accept. e.dan and glenn's answers are the most pragmatic, but Ole Tanges is the most esthetically pleasing. chorobas is also quite nice. Have to think about it. — Alexander Torstling, Jan 30 '12 at 13:04

score 3 · Accepted Answer · answered Jan 29 '12 at 17:19

If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:

find . -name '*.txt' | parallel grep -il iso-8859-1 | parallel iconv -f ISO-8859-1 -t UTF-8 {} \> {}.converted

You can install GNU Parallel simply by:

wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem

Watch the intro videos for GNU Parallel to learn more: https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1

score 2 · Answer 2 · answered Jan 24 '12 at 11:36

2

How about a for loop like:

for file in `find . -name '*.txt' | xargs grep 'iso-8859-1' | cut -d ':' -f1`; do
    iconv -f ISO-8859-1 -t UTF-8 $file > $file.converted
done

answered Jan 24 '12 at 11:36

e.dan

7,275
1
26
29

How will your solution deal with the file: `My brother's 12" records: Listed a->z.txt`? – Ole Tange Aug 11 '14 at 22:38

glenn jackman · Answer 3 · 2012-01-24T16:23:04.690

1

Assuming none of your files have newline characters in the name, and assuming you have GNU find and xargs::

find . -name '*.txt' -print0 |
xargs -0 grep -l 'iso-8859-1' |
while read -r file; do
    iconv -f ISO-8859-1 -t UTF-8 "$file" > "$file".converted 
done

With grep -l, you don't need the cut command in the pipeline.

edited Jan 24 '12 at 16:23

answered Jan 24 '12 at 16:16

glenn jackman

238,783
38
220
352

score 0 · Answer 4 · answered Dec 21 '14 at 13:34

echo the command you want xargs to operate on to a string that is piped to the shell and that will overcome the substitution problem.

find . -name '*.txt' | xargs grep 'iso-8859-1' | cut -d ':' -f1 | 
xargs echo "iconv -f ISO-8859-1 -t UTF-8 {} > {}.converted" | bash

score 0 · Answer 5 · answered Jan 24 '12 at 17:23

0

You are almost there:

find . -name '*.txt' | xargs grep -i iso-8859-1 | cut -f1 -d: | \
xargs -I% echo iconv -f l1 -t utf8 % \> %.utf | bash

answered Jan 24 '12 at 17:23

choroba

231,213
25
204
289

How will your solution deal with the file: `My brother's 12" records: Listed a->z.txt`? – Ole Tange Aug 11 '14 at 22:36

xargs: variable substitution after redirection

5 Answers5