1

First post here so go easy on me. I'm new to Python.

I'm trying to download a .zip file from a website that requires a login to access the link. In the Zip file is a .csv. I've tried a few different methods but can't get the file to download. This doesn't throw any errors but the file doesn't appear. Sorry i can't give out the password and username.

Am i missing something? Do i need a different way to do it?

Thanks for your help.

UPDATE: still no joy. I managed to do it with VBA in excel but not python. Any ideas?

import requests
from requests.auth import HTTPBasicAuth

theurl= 'https://www.pencarrie.com/export/products.zip'
username = 'Secret'
password = 'Secret'
filename = os.path.basename(urlparse(theurl).path)
r=requests.get(theurl, auth=HTTPBasicAuth(username, password))

I've now switched to a different approach and am still getting the same result. The file that is downloaded is now complete. or is just the zip generated by the write command. the download complete command says its complete.......HELP PLEASE.

    import requests

userid = '???????'
password = '??????'


site_url = 'https://www.pencarrie.com/marketing/website-options/data/enhanced-data'
file_url = 'https://www.pencarrie.com/export/products.zip'


o_file = 'products.zip'  

# create session
s = requests.Session()
# GET request. This will generate cookie for you
s.get(site_url)
# login to site.
s.post(site_url, data={'_username': userid, '_password': password})
# Next thing will be to visit URL for file you would like to download.
r = s.get(file_url)

# Download file
with open(o_file, 'wb') as output:
    output.write(r.content)
print(f"requests:: File {o_file} downloaded successfully!")

# Close session once all work done
s.close()

I have it running with VBA so I assume it's basic authentication but it just downloads html. But I'm not sure.

John Glanville
  • 15
  • 7

1 Answers1

0

requests.get does not save whatever it downloads to file, instead it saves it to content field of the Response object.

What you need is to write it to file manually:

f = open(filename, "wb")
f.write(r.content)
f.close()
Sergey Kozharinov
  • 64
  • 2
  • 6
  • Hi Sergey, I understand this, When i run the write part, the Zip file isn't complete. It should be 2.5MB but is only 270kb. – John Glanville Feb 25 '22 at 18:59
  • Check the file. There might be an error. Also check r.status_code – Sergey Kozharinov Feb 26 '22 at 07:09
  • The response is 200, as expected. when i use the link in chrome it downloads correctly. (username & password saved in chrome). I think it must be a validation thing. can't work it out. I've managed to do it using VBA in excel but want to do it with Python. – John Glanville Feb 26 '22 at 11:34
  • Try setting User-Agent to 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.99 Safari/537.36' as described in https://stackoverflow.com/a/44535317/18292805 – Sergey Kozharinov Feb 27 '22 at 10:30
  • BTW, are you sure that this website accepts http basic authentication? The url seems to return 200 and 270kb result (login page) without auth at all – Sergey Kozharinov Feb 27 '22 at 10:54