How do I read the size of an archive on web

Question

I want to download a file from a website using python (urllib or urllib2)   While downloading I want to show the percentage of the download.

#!usr/bin/env python
#-*- coding: latin1 -*-

import urllib

url = 'http://ww1.microchip.com/downloads/en/DeviceDoc/pk2cmdv1-20Linux2-6.tar.gz'
urllib.urlretrieve(url, "pickit.tar.gz")

How could I implement this?

Answer 1

The following code should work:

def download(url, local_file):
    def progress(blocks_read, block_size, total_size):
        percent = min(int(blocks_read * block_size * 100 / total_size), 100)
        sys.stdout.write("\rdownloading {}: {:2}%".format(local_file, percent))
        if percent == 100:
            sys.stdout.write('\n')
        sys.stdout.flush()

    urllib.urlretrieve(url, local_file, reporthook=progress)


# call download function
download("http://ww1.microchip.com/downloads/en/DeviceDoc/pk2cmdv1-20Linux2-6.tar.gz",
         "pickit.tar.gz")

Answer 2

Use "urllib2.urlopen(..)" instead and read the headers, it will contain the data you're looking for (content length)!

A good example: Python: Get HTTP headers from urllib2.urlopen call?

Answer 3

This answer has an nice snippet.

In short:

1. get the content-length header
2. read a chunk of data
3. calc percentage
4. repeat 2 and 3 until done