0

I'm using the below command in Terminal on a Mac to read a file of email addresses and convert them to a MD5 hash. 

tr -d " " < em.txt | tr '[:upper:]' '[:lower:]' | while read line; do 
(echo -n $line | md5); done | awk '{print $1}' > hashes1.txt

This produces a file of hashes that are 1 row shorter than the original input file. But I can't figure out why.

This code does a few things, below.

  1. Converts an email address to all lower case
  2. Converts the email address to a MD5 Hash
  3. Outputs a list of new email addresses to a hashes1.txt file

Thanks in advance!

Tali R
  • 1

1 Answers1

1

Your command is wrong : it should be :

tr -d " " < em.txt |
    tr '[[:upper:]]' '[[:lower:]]' |
    while IFS= read -r line; do 
        echo -n "$line" | md5 | awk '{print $1}' >> hashes1.txt
    done

or

while IFS= read -r line; do 
    echo -n "$line" | md5 | awk '{print $1}' >> hashes1.txt
done < <(tr -d " " < em.txt | tr '[[:upper:]]' '[[:lower:]]')

Changed the file feeding place too.

And ensure your file don't have strange characters with 

od -c file

if yes, install dos2unix, then :

dos2unix file

or using :

perl -i -pe 's/\r//g' file
Mévatlavé Kraspek
  • 552
  • 2
  • 15
  • Thanks. When I use that script, it still ends up 1 row shorter. But this time it cuts off the last row. I can tell it's cutting off the last row because I remove "| md5" to check and the last email is cut off. – Tali R Aug 29 '18 at 19:25
  • Try `od -c file` to see if you don't have strange characters, like windows line endings `\r\n` : change it if it's the case : `dos2unix file` – Mévatlavé Kraspek Aug 29 '18 at 19:32
  • Yes, you are right. The file has \r\n. But when I try to use dos2unix I get the below error"-bash: dos2unix: command not found". Any suggestions on what to do? – Tali R Aug 29 '18 at 20:33
  • Or use : `perl -i -pe 's/\r//g' file` – Mévatlavé Kraspek Aug 29 '18 at 20:36
  • Thank you. Unfortunately I get this error when I try to install brew "-bash: brew: command not found". But one other question. If I remove '\r\n', then will my text file become 1 long text string instead of separate email addresses? – Tali R Aug 29 '18 at 20:38
  • Perl command works great, and the file no longer as \r characters. But, the script still drops the last row in the output, unfortunately. So my output file is always 1 row shorter than the input file with the hashing. – Tali R Aug 29 '18 at 21:05
  • Without seeing the file, I can't do nothing more – Mévatlavé Kraspek Aug 29 '18 at 21:07
  • Can you test my new snippet from my answer ? Is it better ? – Mévatlavé Kraspek Aug 29 '18 at 21:08
  • Tested it. Unfortunately, it still cuts off the last line. :( – Tali R Aug 29 '18 at 21:35
  • `while read line` reads up to a newline. When the last line is without a newline, it will be skipped. Fixes: https://stackoverflow.com/a/31397497/3220113 (also use `IFS=` and `read -r` for other reasons) – Walter A Aug 30 '18 at 06:06
  • You didn't add `|| [ -n "$line" ]` for the last incomplete line, but an upvote for all your effort. – Walter A Aug 30 '18 at 09:25