Paramiko: Asynchronous reading for long shell script output using 'select'

Question

I have following code from here with little modification:

#!/usr/bin/env python                                                  

import paramiko                                                        
import select                                                          


server = "192.168.100.100"                                                
port = 22                                                              
name = "root"                                                          
password = "pass"                                                    


def main():                                                            
    client = paramiko.SSHClient()                                      
    client.load_system_host_keys()                                     

    client.set_missing_host_key_policy(paramiko.MissingHostKeyPolicy())
    client.connect(server, port, name, password)                       

    channel = client.get_transport().open_session()                    
    channel.exec_command("/tmp/test.sh")                      

    while True:                                                        
        if channel.exit_status_ready():                                
            break                                                      

        r, w, x = select.select([channel], [], [], 0.0)                
        if len(r) > 0:                                                 
            print channel.recv(1024)                                   

if __name__ == "__main__":                                             
    main()

Where test.sh has following content:

#!/usr/bin/env bash
while true
do
    echo "Message"
    sleep 1
done

So after executing python script CPU usage by script goes up to 100%. It means this select function does not wait until one or more file descriptors are ready for some kind of I/O. As far as knew this is busy loop problem where 'while ... loop' will iterate continuously even data for read is not presented. How I can make it asynchronous read of remote output?

What do you want to happen while `select` is doing it's thing? You have code dependent on the output immediately after. — Eli, Jul 10 '14 at 09:51
For instance if sleep time in bash script is equal to 5. Select.select should wait until output will be generated instead of looping without any sleep. Maybe I misunderstood of meaning 'select'. What I want to do is read generated output asynchronously. — Khamidulla, Jul 10 '14 at 09:54
You're using the word 'asynchronously' but based on your explanation, I don't think that's what you want. It sounds like you're saying you **want** `select` to block until it gets a result. You just don't want to have it loop endlessly, but instead wait without maxing out CPU. Is that right? — Eli, Jul 10 '14 at 10:09

score 3 · Accepted Answer · answered Jul 11 '14 at 17:02

Your problem is you have timeout set to 0 seconds on select, so it doesn't block at all. The default is to block as long as necessary, so either take out the timeout parameter from select, or change it to something larger:

r, w, x = select.select([channel], [], [])

or

r, w, x = select.select([channel], [], [], 10.0)

You can see the difference both in CPU, and also by putting a simple print statement after your while true. In the 0.0 second timeout statement, you'll see it hit continuously. In the greater timeout case you'll see it hit exactly once (and cpu much lower).

Paramiko: Asynchronous reading for long shell script output using 'select'

1 Answers1