2

I make a POST request to a server to generate a CSV file and response of the POST request is CSV data that I want to write to a file.

I can never know the size of the CSV file (it can be 10MB, 100MB or 1000MB) and as such there is no content-length header.

I have written a function that downloads makes a POST request to the server, generates a CSV file and writes the response to a CSV file. However, I am struggling with progress bar.

How do I add a progress bar?

r = requests.post(URL, data, stream=True)
#Download progress here
user6037143
  • 516
  • 5
  • 20
  • Have a look at: [progress-bar-while-download-file-over-http-with-requests](https://stackoverflow.com/questions/37573483/progress-bar-while-download-file-over-http-with-requests) , or: [python-progress-bar-and-downloads](https://stackoverflow.com/questions/15644964/python-progress-bar-and-downloads) – t.m.adam Aug 01 '17 at 14:10
  • I've looked at those but could not figure how to add progress bar in my case since my response headers won't have Content-Length header. – user6037143 Aug 01 '17 at 21:25

1 Answers1

3
def download_file(url, local_path="./"):
    local_filename = url.split('/')[-1]
    path = local_path + local_filename

    r = requests.get(url, stream=True)
    total_size = int(r.headers.get('content-length', 0))

    with open(local_filename, 'wb') as f:
        for chunk in tqdm(r.iter_content(32*1024), total=total_size,unit='B', unit_scale=True):
            if chunk:
                f.write(chunk)

    return path
Jess Yuan
  • 1,631
  • 2
  • 12
  • 16
  • you should also use `unit_divisor=1024` - otherwise it will display Mebibyte instead of Megabyte (but still annotated as MB). – MCO Oct 21 '18 at 04:13