0

I've followed the grequests usage example, but I'm trying to add some progress feedback. A percentage of the completed requests. How could I achieve that?

import grequests

urls = [
    'http://www.heroku.com',
    'http://python-tablib.org',
    'http://httpbin.org',
    'http://python-requests.org',
    'http://kennethreitz.com'
]

def feedback(r, **kwargs):
            print "%s fetched." % r.url
            return r

rs = (grequests.get(u, callback=feedback) for u in urls)
res = grequests.map(rs)
whitenoisedb
  • 403
  • 1
  • 5
  • 16
  • check out: http://stackoverflow.com/questions/3173320/text-progress-bar-in-the-console for how to make the feedback, but the real problem is that the call to .map blocks. – domoarigato Jan 04 '16 at 20:31

2 Answers2

1

Try this:

from gevent import monkey
monkey.patch_all()
import gevent
import sys
import requests

rs = [gevent.spawn(requests.get, u) for u in urls]
[i.start() for i in rs]
while 1:
    gevent.sleep()
    percent = 0.0
    for i in rs:
        if i.successful():
            percent += 100 / len(rs)
    sys.stdout.write(('='*int(percent))+(''*(100-int(percent)))+("\r [ %d"%percent+"% ] "))
    sys.stdout.flush()
    if percent == 100:
        sys.stdout.write('\n')
        sys.stdout.flush()
        break
domoarigato
  • 2,802
  • 4
  • 24
  • 41
0

i am not sure grequests but you can do similar things with another module called pySmartDL.also this module gives a download bar by default. for more info check this answer

Jishnu
  • 106
  • 9