1

I am trying to get some videos downloaded from the website. So when I enter the url in the browser, the webpage will ask me for a username and password with a little window popped out. After I enter them there, the file automatically starts to download. But I got 404 when I tried to use requests.get with an auth parameter to do the same thing.

Code:

url = video_links[0]
print("url: ", url)

filename = os.path.basename(urlparse(url).path)
print('filename: ', filename)

r = requests.get(url, auth=(username, password))
print(r.status_code)

Output:

url:  http://datasets.d2.mpi-inf.mpg.de/movieDescription/protected/avi/0001_American_Beauty/0001_American_Beauty_00.00.51.926-00.00.54.129.avi

filename:  0001_American_Beauty_00.00.51.926-00.00.54.129.avi

404

popped window

I have tried to pass a user-agent in. Still not working.

Code ver2:

headers = {
    'User-Agent': 'Mozilla/something more',
}
with requests.Session() as s:
    r = s.get(url, headers=headers)
    print(r.content)

output:

b'<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">\n<html><head>\n<title>401 Unauthorized</title>\n</head><body>\n<h1>Unauthorized</h1>\n<p>This server could not verify that you\nare authorized to access the document\nrequested.  Either you supplied the wrong\ncredentials (e.g., bad password), or your\nbrowser doesn\'t understand how to supply\nthe credentials required.</p>\n<hr>\n<address>Apache/2.4.38 (Debian) Server at datasets.d2.mpi-inf.mpg.de Port 443</address>\n</body></html>\n'
Invoker
  • 21
  • 4
  • hi, is [`HTTPBasicAuth`](https://docs.python-requests.org/en/master/user/authentication/) failing? Perhaps set your [user agent](https://stackoverflow.com/questions/10606133/sending-user-agent-using-requests-library-in-python) to a known browser as well – jspcal Sep 02 '21 at 19:06
  • Please post your code and out as text. Screenshots of code can not be accepted on Stack Overflow. – Klaus D. Sep 02 '21 at 19:10
  • show real URL for this page. How do you send login and password? Maybe browser sends it in different way. OR maybe browser uses some JavaScript for redirect - but `requests` can't run JavaScript. – furas Sep 03 '21 at 11:35

1 Answers1

1

Just like jspcal said, everything is working now. Thanks

r = requests.get(url, headers=headers, auth=HTTPBasicAuth('username', 'password'))
Invoker
  • 21
  • 4