2

I am using requests library of python to download a file of size approx. 40mb. But with my code i am getting file of size 14mb only. It is not showing any error(few warnings though before download file).

here is my code:

import requests
file_url = "https://file_url.tar"

user='username'
passw='password'
r = requests.get(file_url,auth=(user,passw),verify=False, stream = True)
with open("c.tar","wb") as code:
   for chunk in r.iter_content(chunk_size=1024):
       if chunk:
         code.write(chunk)

I tried using without 'stream=True' too. but that also not working.

When i am puting this URL in browser i am getting complete file of 40 mb.

I tried this script on some other machine and it is working fine there(and i am getting those warnings here too).

These are the warnings i am getting:

SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning

InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning

InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html InsecureRequestWarning)

but i don't think there is a problem because of this because if i am running this script on some other system i am getting these warnings but script is working fine.

akshay Sharma
  • 21
  • 1
  • 3
  • i have a similar code, but i use chunk_size=8, you could give it a try, but i don't know if it'll help. – Avishay Cohen Dec 14 '17 at 10:38
  • what warnings ? always put full error message (Traceback) and warnings in question (as text, not screenshot). – furas Dec 14 '17 at 10:40
  • if there are some problems on network then it can loose connection. If you would know file size before donwloading then you would know if you have to repeat downloading. You should have file size in header `content-size` – furas Dec 14 '17 at 10:43
  • BTW: [Get file size using python-requests, while only getting the header](https://stackoverflow.com/questions/14270698/get-file-size-using-python-requests-while-only-getting-the-header) and [How to resume file download in Python?](https://stackoverflow.com/questions/22894211/how-to-resume-file-download-in-python) – furas Dec 14 '17 at 10:47
  • Try it https://stackoverflow.com/questions/16694907/how-to-download-large-file-in-python-with-requests-py – Serhii Dec 14 '17 at 13:13
  • @asergey OP uses amoste the same code as in marked answer in your link. OP has even `stream=True`. I don't see difference – furas Dec 14 '17 at 13:40
  • can you add url to this file,so we could test it ? – furas Dec 14 '17 at 14:13

2 Answers2

1

I am using urllib instead of requests 

import urllib
url = "http://62.138.7.10/downloads/Bh2g2m.Bh2g.06.DR.M0viesC0unter.mp4?st=6MVZyTUL7X22v7ILOtB2XA&e=1502823147"
file_name = 'trial_video.mp4'
response = urllib.urlopen(url)
with open(file_name,'wb') as f:
    f.write(response.read())

Hope this will help you

babygame0ver
  • 447
  • 4
  • 16
0

I have experienced similar problems with Requests. Requests is great for doing fancy JSON api POST requests etc, but for ordinary file downloads, pycurl is a much better tool. The complicated dependency on libcurl means you shouldn't try installing pycurl with pip; instead you need to download a copy from your distro, or use one of the prebuilt win32 modules from their site.

For what it's worth, when I was using requests for file downloads, I also set up logging, and I got some "broken pipe" errors. Maybe Requests disconnects early for performance reasons or something? I didn't have the patience to figure it out when I knew there was an alternative solution that works reliably.

brotatochip
  • 11
  • 2