Passing username and password with urllib3 in python

Question

I am trying to obtain the html response from the following page

https://ghrc.nsstc.nasa.gov/pub/lis/iss/data/science/nqc/nc/2020/0101/

When I open this url in chrome, I have to enter username an password for the account I have with the site

I want to pass this username and password using urllib3 in python, my current code is

import urllib3

url = 'https://ghrc.nsstc.nasa.gov/pub/lis/iss/data/science/nqc/nc/2020/0101/'
username = ''
password = ''
data = {'Username': username, 'Password': password}

http = urllib3.PoolManager()
r = http.request('POST', url, data)
print(r.status)

print(r.data)

However running this still gives the response of the login page

I am not sure if i need to use cookies, or how to figure out what format the username and password need to be passed to the url for login to succeed and be taken to the url specified

Maybe these two questions can be useful to you? [Login on a site using urllib](https://stackoverflow.com/questions/29048168/login-on-a-site-using-urllib) and [Python urllib3 login search](https://stackoverflow.com/questions/29061135/python-urllib3-login-search). — mrodo, Feb 27 '20 at 23:32
Does this answer your question? [Login on a site using urllib](https://stackoverflow.com/questions/29048168/login-on-a-site-using-urllib) — mrodo, Feb 27 '20 at 23:32
As an aside, why use urllib directly instead of something like requests? — AMC, Feb 27 '20 at 23:47
@matteo.rebeschi There’s also [this question](https://stackoverflow.com/questions/11335825/authentication-with-urllib3/11388614). — AMC, Feb 27 '20 at 23:48

score 0 · Answer 1 · answered Feb 27 '20 at 23:38

It'll be difficult to do this with a pure POST request, at least for me. For a project like this I would use Selenium

pip install selenium

Download Chrome driver from here: https://sites.google.com/a/chromium.org/chromedriver/downloads

From downloaded files, copy the chromedriver.exe file to the app root.

And here is the code to login into https://ghrc.nsstc.nasa.gov/pub/lis/iss/data/science/nqc/nc/2020/0101/

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
#create an instance of webdriver
driver = webdriver.Chrome()

#navigate to URL
driver.get("https://ghrc.nsstc.nasa.gov/pub/lis/iss/data/science/nqc/nc/2020/0101")

# username and password variable
username = 'my_username'
password = 'my_password'

#get the username and password fields by id and fill them
input_user = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'username')))
input_user.send_keys(username)
input_pwd = driver.find_element_by_id('password')
input_pwd.send_keys(password)
#click the login button
btn = driver.find_element_by_xpath('//input[@type="submit"]')
btn.click()

Passing username and password with urllib3 in python

1 Answers1