12

I need a progress to show during file download for Python 3. I have seen a few topics on Stackoverflow, but considering that I'm a noob at programming and nobody posted a complete example, just fractions of it, or the one that I can make work on Python 3, none are good for me...

additional info:

ok, so i have this:

from urllib.request import urlopen
import configparser
#checks for files which need to be downloaded
print('    Downloading...')
file = urlopen(file_url)
#progress bar here
output = open('downloaded_file.py','wb')
output.write(file.read())
output.close()
os.system('downloaded_file.py')

script is run through python command line

Cœur
  • 37,241
  • 25
  • 195
  • 267
Mirac7
  • 1,566
  • 4
  • 26
  • 44
  • 5
    This question is missing a lot of helpful information: Where do you want this progress bar? In the console? On a web page? In some desktop app? How are you downlading the file? etc etc – Alex Couper Dec 14 '12 at 15:16
  • (urllib2 in Python 2 is more or less the same as urllib in Python 3) – poke Dec 14 '12 at 19:42
  • i noticed that urllib.request in python 3 is what urllib2 is python 2... correct me if i'm wrong... – Mirac7 Dec 14 '12 at 19:48
  • Related: [Text Progress Bar in the Console](http://stackoverflow.com/q/3173320/4279) – jfs Feb 07 '17 at 16:43
  • 1
    @Cœur the link is useful (for those who need Python 2). You could move the link into the comments if you'd like¶ Python 3 code can be improved (and therefore it is not useful to prevent future answers). – jfs Jul 08 '18 at 17:08
  • Related: [Python urllib2 Progress Hook](https://stackoverflow.com/questions/2028517/python-urllib2-progress-hook) – Cœur Jul 08 '18 at 17:10

2 Answers2

25

There is urlretrieve() that downloads an url to a file and allows to specify a reporthook callback to report progess:

#!/usr/bin/env python3
import sys
from urllib.request import urlretrieve

def reporthook(blocknum, blocksize, totalsize):
    readsofar = blocknum * blocksize
    if totalsize > 0:
        percent = readsofar * 1e2 / totalsize
        s = "\r%5.1f%% %*d / %d" % (
            percent, len(str(totalsize)), readsofar, totalsize)
        sys.stderr.write(s)
        if readsofar >= totalsize: # near the end
            sys.stderr.write("\n")
    else: # total size is unknown
        sys.stderr.write("read %d\n" % (readsofar,))

urlretrieve(url, 'downloaded_file.py', reporthook)

Here's a GUI progress bar:

import sys
from threading import Event, Thread
from tkinter import Tk, ttk
from urllib.request import urlretrieve

def download(url, filename):
    root = progressbar = quit_id = None
    ready = Event()
    def reporthook(blocknum, blocksize, totalsize):
        nonlocal quit_id
        if blocknum == 0: # started downloading
            def guiloop():
                nonlocal root, progressbar
                root = Tk()
                root.withdraw() # hide
                progressbar = ttk.Progressbar(root, length=400)
                progressbar.grid()
                # show progress bar if the download takes more than .5 seconds
                root.after(500, root.deiconify)
                ready.set() # gui is ready
                root.mainloop()
            Thread(target=guiloop).start()
        ready.wait(1) # wait until gui is ready
        percent = blocknum * blocksize * 1e2 / totalsize # assume totalsize > 0
        if quit_id is None:
            root.title('%%%.0f %s' % (percent, filename,))
            progressbar['value'] = percent # report progress
            if percent >= 100:  # finishing download
                quit_id = root.after(0, root.destroy) # close GUI

    return urlretrieve(url, filename, reporthook)

download(url, 'downloaded_file.py')

On Python 3.3 urlretrieve() has different reporthook interface (see issue 16409). To workaround it, you could access the previous interface via FancyURLopener:

from urllib.request import FancyURLopener
urlretrieve = FancyURLopener().retrieve

To update the progress bar within the same thread, you could inline urlretrieve() code:

from tkinter import Tk, ttk
from urllib.request import urlopen

def download2(url, filename):
    response = urlopen(url)
    totalsize = int(response.headers['Content-Length']) # assume correct header
    outputfile = open(filename, 'wb')

    def download_chunk(readsofar=0, chunksize=1 << 13):
        # report progress
        percent = readsofar * 1e2 / totalsize # assume totalsize > 0
        root.title('%%%.0f %s' % (percent, filename,))
        progressbar['value'] = percent

        # download chunk
        data = response.read(chunksize)
        if not data: # finished downloading
            outputfile.close()
            root.destroy() # close GUI
        else:
            outputfile.write(data) # save to filename
            # schedule to download the next chunk
            root.after(0, download_chunk, readsofar + len(data), chunksize)

    # setup GUI to show progress
    root = Tk()
    root.withdraw() # hide
    progressbar = ttk.Progressbar(root, length=400)
    progressbar.grid()
    # show progress bar if the download takes more than .5 seconds
    root.after(500, root.deiconify)
    root.after(0, download_chunk)
    root.mainloop()

download2(url, 'downloaded_file.py')
jfs
  • 399,953
  • 195
  • 994
  • 1,670
  • @Mirac7: I've added variant that shows GUI and download the file from within the same thread. – jfs Dec 16 '12 at 17:32
  • 1
    Good and complete answer +1 – doru Jan 12 '13 at 09:05
  • @J.F.Sebastian Is there a possible way to include download speed in a `reporthook`? – SvbZ3r0 Jun 09 '16 at 12:04
  • @GughanRavikumar yes. Store the time you've received the last block and divide it by a block size to get the current download speed. – jfs Jun 09 '16 at 14:08
  • @J.F.Sebastian Silly me!! I didn't think of that. Thank you so much! – SvbZ3r0 Jun 09 '16 at 15:23
  • 1
    @GughanRavikumar it could look like: `t = time.monotonic(); speed = blocksize / (t - last[0]); average_speed = readsofar / (t - start); last[0] = t` – jfs Jun 09 '16 at 16:22
  • @J.F.Sebastian I got it working with something similar. Thank you for your time. – SvbZ3r0 Jun 09 '16 at 16:42
2

I think this piece of code can help you. I'm not quite sure it's exactly what you want. At least it should give you something to work on.

import tkinter 
from tkinter import ttk
from urllib.request import urlopen


def download(event):
    file = urlopen('http://www.python.org/')
    output = open('downloaded_file.txt', 'wb')
    lines= file.readlines()
    i = len(lines)

    for line in lines:
        output.write(line)
        pbar.step(100/i)

    output.close()
    file.close()




root = tkinter.Tk()
root.title('Download bar')

pbar = ttk.Progressbar(root, length=300)
pbar.pack(padx=5, pady=5)

btn = tkinter.Button(root, text="Download")
# bind to left mouse button click
btn.bind("<Button-1>", download)
btn.pack(pady=10)

root.mainloop()

This works, I've tried it.

doru
  • 9,022
  • 2
  • 33
  • 43
  • I'm recieving error io.UnsupportedOperation: seek... Am I doing something wrong? – Mirac7 Dec 15 '12 at 15:03
  • I think seek() method doesn't work with urlopen().Remove the line file.seek(0) and see the results. – doru Dec 15 '12 at 15:15
  • already tried... it works fine, but when i click download, it simply creates empty script, and progressbar is empty. – Mirac7 Dec 15 '12 at 16:02
  • 1
    @doru To avoid opening the URL twice, why not assign `file.readlines()` to a variable? – Benjamin Hodgson Dec 15 '12 at 16:35
  • @poorsod You're right! Thanks a lot! I've already (re)edited the answer. +1 for this. – doru Dec 15 '12 at 16:52
  • can you tell me how to make percent progress displayed through progress bar? – Mirac7 Dec 15 '12 at 17:53
  • I don't know if this is possible but I'll find out and let you know. – doru Dec 15 '12 at 17:56
  • thanks... i have only one file to be downloaded, so i can't make use of progress bar like the one in your example in this specific situation... – Mirac7 Dec 15 '12 at 18:08
  • @doru: you could get an approximate file size from `Content-Length` header as urlretrieve() does. It will allow you to show the percent progress. – jfs Dec 15 '12 at 20:09