Popen with Threads - Not Collecting All Output

Question

I'm trying to write a Python tool that can ping multiple hosts, using multiple threads. I have my code mainly working. Code below...

from threading import Thread
from subprocess import Popen, PIPE

class Ping:
    def __init__(self, hosts):
        self.hosts = hosts
        self.results = dict()

    def register_threads(self):
        self.threads = list()
        for host in self.hosts:
            print('registering thread %s' % host)
            thread = Thread(target=self.run_command, args=(host,))
            self.threads.append(thread)

    def start_threads(self):
        for thread in self.threads:
            thread.start()

    def main(self):
        self.register_threads()
        self.start_threads()

    def run_command(self, host):
        process = Popen('ping {} -c 50'.format(host), stdout=PIPE, shell=True)
        self.results[host] = list()
        while True:
            line = process.stdout.readline().rstrip()
            print(line)
            if not line:
                print("not line")
                break
            self.results[host].append(str(line, 'utf-8'))

However, if the host is not reachable I do not get anything returned? i.e the Like so,

>>> from lib.ping import *
>>> ping = Ping(hosts=['1.2.2.2'])
>>> ping.main()
registering thread 1.2.2.2
>>> print(ping.results)
{'1.2.2.2': ['PING 1.2.2.2 (1.2.2.2) 56(84) bytes of data.']}

However, from a ping on another Liunx device I get,

rick@abc:~$ ping 1.2.2.2 -c 5
PING 1.2.2.2 (1.2.2.2) 56(84) bytes of data.

--- 1.2.2.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4004ms

Any ideas of how to get the statistics line, and also know if the host is not reachable.

Thanks.

Could you try with a `ping {} -c 50 -W1` as the parameter in `Popen`? — Kunal Gupta, Apr 28 '19 at 21:22
Possible duplicate of [How to while loop until the end of a file in Python without checking for empty line?](https://stackoverflow.com/questions/29022377/how-to-while-loop-until-the-end-of-a-file-in-python-without-checking-for-empty-l) — Kent Shikama, Apr 28 '19 at 21:27
^ Or in short, remove the `.rstrip()` and I think you can manage on your own from there. — Kent Shikama, Apr 28 '19 at 21:32

score 0 · Answer 1 · answered Apr 28 '19 at 21:34

If you look at the output you'll see that there's a newline in the actual ping output.

PING 1.2.2.2 (1.2.2.2) 56(84) bytes of data.
<This one>
--- 1.2.2.2 ping statistics ---
5 packets transmitted, 0 received, 100% packet loss, time 4004ms

Your line = process.stdout.readline().rstrip() causes the newline to be ignored and the if not line: causes it to break at that empty line.

So you can either remove the rstrip() call or better use

for line in process.stdout:
    print(line)
    self.results[host].append(str(line, 'utf-8'))

Also if you want a faster response (for example when ping is being blocked and the response is too long) set a timeout using the -W flag and set a timeout for the ping request.

Popen with Threads - Not Collecting All Output

1 Answers1