bash scrip to convert IPs to FQDN only works on last entry

Question

I have the following bash script which reads a text file of IP addresses and then runs the host command and pulls out just the fqdn and drops the period on the end. The command works for every IP in my text file but when I try and run this only the last IP works?

#!/bin/bash
cat hosts.txt | while read -r output
do 
    host $output | awk 'NR==1{print $5}' | sed 's/.$//'
done > results.txt

Here is the output I get

3(NXDOMAIN
3(NXDOMAIN
3(NXDOMAIN
3(NXDOMAIN
3(NXDOMAIN
dsfprmtas07.mydomain.com

hosts.txt turned out the be the issue hosts2.txt works but not sure why

Add output of `head -n 1 hosts.txt | hexdump -C` to your question (no comment). — Cyrus, Oct 23 '20 at 22:16
Shouldn't this be enough? `cat hosts.txt | xargs -L1 host | awk '{print $5}' | sed 's/.$//'` — Md. Minhazul Haque, Oct 23 '20 at 22:21
I'm pretty sure the problem is in the hosts.txt file; without it (see Cyrus' request), there's no way to tell what that problem is. — Gordon Davisson, Oct 23 '20 at 22:43
@GordonDavisson you were right, it was the text file. Which is odd to me because I could print the line by line and it only showed the IP each time. I created a new text file and copied the IPs and it worked. I will attach both files, maybe someone can help me understand why hosts2.txt works but hosts.txt only worked on the last IP. — dlevens, Oct 23 '20 at 23:16
`cat -vet hosts.txt | tail -10`, if you see `^M$` at the end of each line, then your file is MS-DOSified, fix it with `unix2dos hosts.txt`. Good luck. — shellter, Oct 23 '20 at 23:28
The hosts.txt file has DOS/Windows line endings on all but the last line, which include a carriage return as well as the linefeed that unix tools expect. See [this Q](https://stackoverflow.com/questions/39527571/are-shell-scripts-sensitive-to-encoding-and-line-endings) for more info. — Gordon Davisson, Oct 23 '20 at 23:29

Cyrus · Answer 1 · 2020-10-23T23:32:33.623

1

Replace

cat hosts.txt

with

dos2unix < hosts.txt

to get rid of your carriage returns.

edited Oct 23 '20 at 23:32

answered Oct 23 '20 at 22:17

Cyrus

84,225
14
89
153

score 1 · Accepted Answer · answered Oct 24 '20 at 00:00

You can setup the Internal Fields Separator environment variable IFS to accept either Unix or DOS newlines regardless:

#!/usr/bin/env bash

# Accept Unix or DOS newlines regardless as record delimiters
IFS=$'\n\r '

while read -r ip_addr
do
  # Read the fully qualified domain name as fifth field
  # by reading first 4 fields into _ placeholders
  read -r _ _ _ _ fqdn < <(host "$ip_addr")

  # Strip trailing dot of FQDN to get the host_name
  host_name="${fqdn%.}"

  # Just print
  echo "$host_name"
done <hosts.txt

bash scrip to convert IPs to FQDN only works on last entry

2 Answers2