0

I have about 700 text files that consist of config output which uses various special characters. I am using this script to remove the special characters so I can then run a different script referencing an SED file to remove the commands that should be there leaving what should not be in the config. I got the below from Remove all special characters and case from string in bash but am hitting a wall.

When I run the script it continues to loop and writes the script into the output file. Ideally, it just takes out the special characters and creates a new file with the updated information. I have not gotten to the point to remove the previous text file since it probably wont be needed. Any insight is greatly appreciated.

for file in *.txt for file in *.txt
do
   cat * | tr -cd '[:alnum:]\n\r' | tr '[:upper:]' '[:lower:]' >> "$file" >> "$file".new_file.txt
done
John Kugelman
  • 349,597
  • 67
  • 533
  • 578
user7915836
  • 3
  • 1
  • 1
    `cat * |` means "send the contents of all the files in the current directory into the pipe." I'm pretty sure that's not what you want to do. – rici Feb 06 '19 at 02:50
  • Also, using multiple `>>`s doesn't make sense. Each redirection changes stdout. The last definition, and *only* that last definition, wins; previous ones might have side effects (like creating files that didn't exist before), but they don't change where your output goes. – Charles Duffy Feb 06 '19 at 02:53
  • @jhnc, three backticks will suffice. Perhaps you were thinking of the number of spaces needed before the new convention was adopted? – Charles Duffy Feb 06 '19 at 02:59
  • @CharlesDuffy no idea where I got 4 backticks from. Poor eyesight maybe. :) – jhnc Feb 06 '19 at 03:10

2 Answers2

1

A less-broken version of this might look like:

#!/usr/bin/env bash
for file in *.txt; do
 [[ $file = *.new_file.txt ]] && continue ## skip files created by this same script
 tr -cd '[:alnum:]\n\r' <"$file" \
   | tr '[:upper:]' '[:lower:]' \
   >> "$file".new_file.txt
done

Note:

  • We're referring to the "$file" variable being set by for.
  • We aren't using cat. It slows your script down with no compensating benefits whatsoever. Instead, using <"$file" redirects from the specific input file being iterated over at present.
  • We're skipping files that already have .new_file.txt extensions.
  • We only have one output redirection (to the new_file.txt version of the file; you can't safely write to the file you're using as input in the same pipeline).
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
0

Using GNU sed:

sed -i 's/[^[:alnum:]\n\r]//g;s/./\l&/g' *.txt
agc
  • 7,973
  • 2
  • 29
  • 50
  • I used the sed option and its working great. I went this route for simplicity because it will be used by many that are not as bash savvy. The sed option works great. Thank you all for your quick response and insight. – user7915836 Feb 06 '19 at 14:25