How to print progress from this code as the subprocess is running?

Question

The code above shows the rsync progress only when the code finishes execution. I want to print the progress as it is happenning. For example when the file is transferring I want to show the progress all along. How can I do that?

    import re 
    import subprocess

    def sort(rprogress):
        '''This function extracts the percentage from the rsync progress and return strings of percentanges'''
        progress1 = re.findall(r'\d+%', rprogress)
        remove_duplicate = set(progress1)     #Remove Duplicate percentage from list
        remove_percent = [a.replace('%', '') for a in remove_duplicate]     #Removes percentage
        sorted_list = sorted(remove_percent, key=int)     #Sort in ascending order
        #result = ', '.join(map(lambda sorted_list: str(sorted_list) + '%', sorted_list)) #Adds the percentage
        return sorted_list


    source12 = 'sachet.adhikari@69.43.202.97:/home/sachet/my_files/ok.txt'
    password = 'password'
    destination = '/home/zurelsoft/files'
    result = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz', '--info=progress2', source12, destination], 
                                       stdout=subprocess.PIPE).communicate()[0]   

    print result               


print sort(result)

Answer 1

The stdout=subprocess.PIPE prevents the subprocess from printing to stdout and instead passes it to result using communicate similar to how in bash

sshpass -[args] rsync [source] [dest]

will print progress but

sshpass -[args] rsync [source] [dest] | sort

will not print anything until the process is complete.

What you want is to tee the stdout. look here. Based on those answers you could do something like:

# Caution! untested code
result = []
process = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz',
                            '--info=progress2', source12, destination], 
                           stdout=subprocess.PIPE)
while process.poll() is None:
    line = process.stdout.readline()
    print line
    result.append(line)
print sort(result)