How to write a polling function in Python?

Question

I am trying to write a function that reads the variable every 1 minute and return the value of each time. The variable name is proc:

proc = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz', '--info=progress2', source12, destination], 
                                    stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()[0]

The progress is stored in proc variable. I want the function to poll the variable every 1 minute and return value. This is done until the variable is upon execution. What's the best way to do it?

Tried using:

def doWork():
    while True:
        proc = subprocess.Popen(['sshpass', '-p', password, 'rsync', '-avz', '--info=progress2', source12, destination], 
                                    stderr=subprocess.PIPE, stdout=subprocess.PIPE).communicate()[0]stdout=subprocess.PIPE).communicate()[0]
        data = sort(proc)
        print data 
        time.sleep(10)

No luck though! It prints the entire progress at same time and loop over.

You can check this, http://stackoverflow.com/questions/1038907/run-a-task-at-specific-intervals-in-python — Qiang Jin, Dec 24 '12 at 08:59

score 4 · Accepted Answer · edited May 23 '17 at 12:33

The code below will run rsync and read any output from the command the moment it is available. I used the --progress option of rsync to have it print out it's progress. The progress updates sometimes end in a \n and sometimes in a \r. So I read individual characters and then form each line of characters by looking for either of those two characters. Whenever I encounter a new progress line I print it to the screen. You could chose to do anything though such as parsing the percent complete and displaying a graphical progress bar. If you are interested in seeing how to produce a progress bar in terminal check this answer out. I put an example call of the sync function and example output.

code

from subprocess import Popen, PIPE

def sync(src, dest, passwd):
    cmd = ['sshpass', '-p', passwd, 'rsync', '-avz', '--progress', src, dest]
    p = Popen(cmd, stdout=PIPE)
    line = ''
    while True:
        c = p.stdout.read(1)
        if not c:
            break
        if c in ['\n', '\r']:
            print 'rsync progress: %s' % line
            line = ''
        else:
            line += c

sync('/path/big.txt', 'myserver:/path/', 'mypassword')

output

rsync progress: sending incremental file list
rsync progress: big.txt
rsync progress: 
rsync progress:        32768   0%    0.00kB/s    0:00:00  
rsync progress:     65798144  31%   62.72MB/s    0:00:02  
rsync progress:    131596288  62%   62.77MB/s    0:00:01  
rsync progress:    197427200  94%   62.79MB/s    0:00:00  
rsync progress:    209715200 100%   62.80MB/s    0:00:03 (xfer#1, to-check=0/1)
rsync progress: 
rsync progress: sent 204032 bytes  received 31 bytes  45347.33 bytes/sec
rsync progress: total size is 209715200  speedup is 1027.70

Thank you so much for your answer. How do I show the same thing in Django(in web browser?) — user1881957, Dec 25 '12 at 03:43
@user1881957 look at http://stackoverflow.com/q/2922874/1699750. Check out my answer and the other answers too. — Marwan Alsabbagh, Dec 25 '12 at 03:47
That was quite helpful but not too specific to my question. Can you please edit your answer. — user1881957, Dec 25 '12 at 04:07
@user1881957 I recommend post a new question and give more details of exactly what you would like. Namely django, do you want a progressbar? or just text output. The answer above solves the question you posed. Also have a go at it, adapt the code, show us where you get stuck. Generally in stack overflow there isn't a culture of writing the code for you, more trying to help you get through coding issues you face. — Marwan Alsabbagh, Dec 25 '12 at 04:31
Here's the link to the new question: http://stackoverflow.com/questions/14027845/show-the-rsync-progress-using-the-following-code-in-django — user1881957, Dec 25 '12 at 05:04

How to write a polling function in Python?

1 Answers1